{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6HPvYgkOjlHg"
   },
   "source": [
    "# 逻辑回归 Logistic Regression\n",
    "\n",
    "此Notebook是配合Andrew Ng \"Machine Leanring\"中[逻辑回归](https://github.com/loveunk/machine-learning-deep-learning-notes/blob/master/machine-learning/logistic-regression.md)部分学习使用。\n",
    "\n",
    "测试用python版本为3.6\n",
    "* 机器学习路径：https://github.com/loveunk/machine-learning-deep-learning-notes/\n",
    "* 内容正文综合参考网络资源，使用中如果有疑问请联络：www.kaikai.ai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2bSRbRGTjlHi"
   },
   "source": [
    "在这一次练习中，我们将要实现逻辑回归并且应用到一个分类任务。我们还将通过将正则化加入训练算法，来提高算法的鲁棒性，并用更复杂的情形来测试它。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "WXk4cLxBjlHj"
   },
   "source": [
    "## 准备数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "nfrEaVuajlHl"
   },
   "source": [
    "在训练的初始阶段，我们将要构建一个逻辑回归模型来预测，某个学生是否被大学录取。设想你是大学相关部分的管理者，想通过申请学生两次测试的评分，来决定他们是否被录取。现在你拥有之前申请学生的可以用于训练逻辑回归的训练样本集。对于每一个训练样本，你有他们两次测试的评分和最后是被录取的结果。为了完成这个预测任务，我们准备构建一个可以基于两次测试评分来评估录取可能性的分类模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "2zO2vzfujlHm"
   },
   "source": [
    "让我们从检查数据开始。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "meEb58oJjlHn"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('fivethirtyeight')\n",
    "from sklearn.metrics import classification_report # 这个包是评价报告"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zqEIXn3sjlHr"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>exam1</th>\n",
       "      <th>exam2</th>\n",
       "      <th>admitted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>34.623660</td>\n",
       "      <td>78.024693</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>30.286711</td>\n",
       "      <td>43.894998</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>35.847409</td>\n",
       "      <td>72.902198</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>60.182599</td>\n",
       "      <td>86.308552</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>79.032736</td>\n",
       "      <td>75.344376</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       exam1      exam2  admitted\n",
       "0  34.623660  78.024693         0\n",
       "1  30.286711  43.894998         0\n",
       "2  35.847409  72.902198         0\n",
       "3  60.182599  86.308552         1\n",
       "4  79.032736  75.344376         1"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = 'ex2data1.txt'\n",
    "data = pd.read_csv(path, header=None, names=['exam1', 'exam2', 'admitted'])\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8r4bOK-K8J4i"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>exam1</th>\n",
       "      <th>exam2</th>\n",
       "      <th>admitted</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "      <td>100.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>65.644274</td>\n",
       "      <td>66.221998</td>\n",
       "      <td>0.600000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>19.458222</td>\n",
       "      <td>18.582783</td>\n",
       "      <td>0.492366</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>30.058822</td>\n",
       "      <td>30.603263</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>50.919511</td>\n",
       "      <td>48.179205</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>67.032988</td>\n",
       "      <td>67.682381</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>80.212529</td>\n",
       "      <td>79.360605</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>99.827858</td>\n",
       "      <td>98.869436</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            exam1       exam2    admitted\n",
       "count  100.000000  100.000000  100.000000\n",
       "mean    65.644274   66.221998    0.600000\n",
       "std     19.458222   18.582783    0.492366\n",
       "min     30.058822   30.603263    0.000000\n",
       "25%     50.919511   48.179205    0.000000\n",
       "50%     67.032988   67.682381    1.000000\n",
       "75%     80.212529   79.360605    1.000000\n",
       "max     99.827858   98.869436    1.000000"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "bvQFnaZojlH1"
   },
   "source": [
    "让我们创建两个分数的散点图，并使用颜色编码来可视化，如果样本是正的（被接纳）或负的（未被接纳）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "4VZ-TIUmjlH3"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABG4AAALQCAYAAADB3TZVAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAAC8aklEQVR4nOzdfVzV9f3/8ec5B7nwHAS1MAskcVakTrdmF2htXSwY1jDWpWXbWiwRN7uahbPWBck3t7W5gRS0VT+1i1UUJSbWupjkaq6twiJn0gyqSWkgIBdyzvn9QQc5AsfD4VxxzuN+u3UzP+d9znkf/HDg8zzv9+tlaGpqsgsAAAAAAABBxxjoCQAAAAAAAGBgBDcAAAAAAABBiuAGAAAAAAAgSBHcAAAAAAAABCmCGwAAAAAAgCBFcAMAAAAAABCkCG4AAAAAAACCFMENAAAAAABAkCK4AQAAAAAACFIENwAAAAAAAEGK4AYAAAAAACBIEdyEqY6ODtXV1amjoyPQU0EQ4vyAK5wfcIXzA65wfsAVzg+4wvkBV0L9/CC4CWNWqzXQU0AQ4/yAK5wfcIXzA65wfsAVzg+4wvkBV0L5/CC4AQAAAAAACFIENwAAAAAAAEGK4AYAAAAAACBIEdwAAAAAAAAEKYIbAAAAAACAIBUR6AkAAAAAAEYmm82mtrY2n7dhttlsioyMVHNzs1paWnz6XBh5gun8iI6OltlsltHovXUyBDcAAAAAgCGz2Wzau3evLBaLjjrqKBkMBp8+V1dXlyIjI716QYzQECznh91uV0dHh/bu3avx48d7bS6c8QAAAACAIWtra5PFYlFMTIxPQxtgpDAYDIqJiZHFYlFbW5vXHpfgBgAAAAAwZB0dHYqOjg70NICgEx0d7dXtgyMiuHniiSd0/fXX6zvf+Y4SEhIUHx+v9evXDzp+//79Wr58uaZPn66EhARNnz5dy5cv1/79+we9z5NPPqlzzjlHxx57rJKTk3XJJZfo3//+ty9eDgAAAACEBFbaAP15+/tiRAQ3BQUFevjhh1VfX68JEya4HNvW1qZ58+ZpzZo1mjp1qhYvXqyTTjpJa9as0bx58wZcrvTb3/5WOTk5amxs1I9//GNddNFFevPNN5Wenq4tW7b46mUBAAAAAAC4NCKCmz/+8Y969913tWvXLl1zzTUux65evVo1NTVaunSpnnnmGd1xxx166qmntGzZMtXU1Gj16tVO43ft2qXCwkJ97Wtf0+uvv6577rlHv//971VVVaWIiAj9/Oc/V3d3ty9fHgAAAAAAwIBGRHDzne98R5MmTTriOLvdrrVr18pisWjZsmVOt914442Kj4/XunXrZLfbe4+vX79e3d3duummmxQXF9d7PDU1VZdffrk++ugj/e1vf/PeiwEAAAAAAHDTiAhu3LVr1y599tlnOu2002Q2m51ui46OVlpamj799FPV1dX1Hq+urpYknXPOOf0ez3Hs9ddf9+GsAQAAAADhbMuWLYqPj1dhYaHPniM+Pl7z5s3z2eMPhz9e/0gWEegJeNOuXbskSSkpKQPePmXKlN5xff/fYrEMWDun7xh3eLNqtK91dXU5/Qn0xfkBVzg/4ArnB1zh/IArnB8jj81mk81m88tzOXZN2O12vz3nUOXm5uqJJ55QQkKCtm/frogI9y+3/fn6+j5+Xl6eHnvsMb399tsD7nIZN26c5syZo+eff96ncxru6w/G88Nms7nMCIbSkS2kghtH16i+W576io2NdRrn+P+jjz7a7fGufPrpp7JarW7PNxjs2bMn0FNAEOP8gCucH3CF8wOucH7AFc6PkSMyMtLvQdvBgwf9+nzuamlp0XPPPSeDwaDGxkZt3LhRGRkZbt/f8bqsVqvPvqZbtmxRTEyM0+M7rl+7uroGfV6bzebzf2dvvf5gOj86OjoGzRJMJtOgC04GElLBTaAde+yxgZ6C27q6urRnzx5NmDBBkZGRgZ4OggznB1zh/IArnB9whfMDrnB+jDzNzc1++7ey2+06ePCgRo0aFZQtyJ9//nm1t7dryZIlKi4u1uOPP67vf//7bt9/1KhRknou6H31NZ02bVq/YyaTSVJPCDfY8xqNRp//Ow/39Qfj+REdHX3ErtjuCqngZsyYMZJ63kAG0tLS4jTO8f+DpWADjXdlKEudgkVkZOSInDf8g/MDrnB+wBXOD7jC+QFXOD9GjpaWFhmN/imb6tj+YjAY/PacQ7Fu3TpFRkbqpptu0ltvvaUXX3xRjY2NOuaYY5zGtbe3695779WTTz6pL774QikpKVq0aFHv6ovDX198fLzmzJmjsrIy3X777Xr55ZfV1dWltLQ0rVq1Sscff7x27typO+64Q6+//rq6u7t17rnn6je/+U2/nSWOx6qsrJQkzZgxQ/X19ZKkWbNm9Y6bM2eObr31Vl144YWSemq+jhs3rvf24uJiXXnllb1/r6ys1AMPPKB33nlHHR0dSklJ0YIFC7R48eLeYMjT1++uYDw/jEaj197LQiq4cdSk6Vt8uC9HrRrHOMf//+Mf/+hN9480HsHHWFcn2wDLzAY7DgAAAGBksFqlTZsiVFQUqaYmKT5eWrKkSxkZ3TosEwiY9957T//61790wQUXaOzYsbr88sv197//XY899phuuOGG3nE2m01XXHGFXn31VZ188sm6+OKLtW/fPi1fvlxz584d9PGbmpqUkZGhCRMm6IorrtCHH36oqqoq/ec//9Fjjz2m733ve5o5c6auuuoqvf3226qoqFBzc7OeffZZl/POzc3Vo48+qu3bt2vRokW9JUcmTZqkSZMm6ZZbbtG9996rpKQkLViwoPd+M2bM6P3/u+66S/fdd5+OO+44ff/731dsbKy2bt2q2267Tf/85z/1yCOPDPv1IwSDm4kTJ+rNN99UW1ubU2epjo4Obd26VRMnTnTaSzZnzhz94x//0Msvv6wrrrjC6fFefvnl3jEITqPKyxWTk6OOggJ15eb2Ho8sKVH0ihVqLyvTwezsAM4QAAAAgCcaGw3KyjKrocGglpZDqyi2bzcpMdGuioo2JSTYAzjDHmvXrpUkXXbZZZKk+fPn65ZbbtG6deucgpvHHntMr776qs477zw98cQTvatRFi1apLPPPnvQx3/vvfe0ePFirVy5svfYjTfeqD//+c/KyMjQrbfeqtyvroXsdrsuu+wybd68We+8845mzpw56OMuXrxYNTU12r59u3Jzc5WcnOx0e35+vu69915NmjRJ+fn5/e7/yiuv6L777tN3v/tdPfLIIxo9enTvHG666Sb9+c9/VkVFhbKysob1+hFi7cANBoMWLlyo1tZWrVq1yum2++67T01NTVq4cKHTnrcrr7xSERER+u1vf+u0xaq2tlaPP/64Jk+erLPOOstvrwHuc4Q2BqtVMfn5iiwpkdQT2sTk5/ccz8nRqPLyAM8UAAAAwFBYrVJWllm1tSan0EaSWlqMqq01KSvLrED3hunq6tJf/vIXxcfHKz09XVJPs5zMzEzt2rVLr7/+eu/Yxx9/XJK0YsUKpy1E06ZN6w19BmKxWPTLX/7S6djFF18sqafr06JFi3qPGwwGZX/1wfX27duH+epcKy0tlST97ne/6w1tHHP41a9+JYPBoKeffrr3uKevHyNkxc3/+3//T3//+98lSe+//76knlSzurpakjRv3jxdcMEFkqSlS5fqhRde0OrVq/Xuu+9q1qxZ2r59u1588UXNmDFDS5cudXrsr33ta7r11ltVUFCgOXPmKCsrSwcOHNDTTz+tgwcPavXq1UNq4wb/6BvaOMTk5yvq/vtl3L2795gjvJHEyhsAAABghNi0KUINDa6LzNbXG1RVFaHMzG4/zaq/yspK7du3T9dcc41TUd3LL79c5eXlWrduXe8Oju3bt2v06NFO9WQczjjjjN6VO4dLSUlx2k0iqbd2zrRp0/oV43Xc9tlnn3n8utzxz3/+U2azedB5x8TEaOfOnb1/9/T1Y4QEN479gX298cYbeuONNyT17MFzBDdms1kbNmzQvffeq+eee07V1dWaMGGCFi9erFtuuaXfCS9JN998syZNmqSSkhL9+c9/1qhRo3Tqqadq+fLl+uY3v+n7F4ghMdbV9Qttem/rE9o4OMIb66xZ1LwBAAAARoDi4qh+K20O19pqVFFRVECDm3Xr1klSvxUj5557riZMmKCKigrde++9vU1xjjvuuAEfJyEhYdDniI2N7XfMsWLF1W2+bo395Zdfqru7W/fee++gY9ra2nr/39PXjxES3JSUlKjkq20w7oiLi9PKlSud9gAeyaWXXqpLL73Uk+nBz2wpKeooKFDMAPssB9NRUEBoAwAAAIwQzc3utXR2d5wvNDQ06JVXXpGk3m1SAykvL9ePfvQjjRkzRl988cWAYxobG30yR1+KjY2VwWAYtDnQ4ULt9fvTiAhugMM5ChG7E960FxY6FS4GAAAAENzi4twrOuzuOF9Yv369bDabzjjjDH3ta1/rd3tXV5eeeOIJrV27Vj/60Y80ffp0bdmyRW+//Xa/7UKO0iD+5lid42infTij0Tjobd/61rf04osvateuXW51Yg7G1z9SENxgxOrKze1X0+ZwtuRkQhsAAABghMnL61RNjdHldimLxaYlSzr9OKtD7Ha71q9fL4PBoJKSEh1//PEDjqutrdVbb72l999/X5dddpm2bNmigoICp65K7733np544gk/zv6QsWPHSpI++eQTTZ48ecDbP/nkkwHve9111+nFF1/UkiVLtH79eo0bN87p9j179qipqUknnniiJAXl6x8pCG4wYkWWlLgMbaSemjeRJSWENwAAAMAIkpHRrcREu2prBx+TlGRXenpg6tu89tpr+vjjj3XmmWcOGtpIPV2M3333Xa1du1b33HOPnnrqKb300ks688wz9d3vfldffvmlnn76aZ199tmqqqry3wv4yllnnaU//vGPuuGGG5SVlSWz2azExERdcsklvbc/88wzuvrqq/X1r39dJpNJ559/vqZNm6bzzjtPv/jFL/TrX/9a3/jGN3TeeecpKSlJ+/btU11dnf7+979rxYoVvcHNggULgu71jxQh1Q4c4cPR8tsdfVuFAwAAAAh+JpNUUdGm1FSrLBbnrToWi02pqVZVVLSpT1dpv3J0QLrqqqtcjrvkkksUGRmpv/zlL+ru7tajjz6qpUuXqqmpSffff7/+8Y9/6J577tGSJUv8Me1+vvvd7+quu+6SzWbT6tWrdeedd+rhhx/uvf3//u//dNFFF+n111/XypUrdeedd+rtt9/uvf2Xv/ylnn32WZ1xxhl67bXXVFxcrKqqKnV1denWW2/tDYCknm1Xwfb6RwpDU1NT4DYFImA6OjpUX1+vpKQkRUdHB3o6Q2Ksq5Nl9uwBu0oNxm4yqXXbNgoUu2kknx/wPc4PuML5AVc4P+AK58fI8/nnn+voo4/26XNYrVJVVYSKiiLV1CTFx0tLlnQpPb07YKENgo/NZlNXV5ciIyNlNAbH+hRvfn+wVQojji0lRe1lZQO2BLclJ/fbPmU3mdReVkZoAwAAAIwwJpOUmdmtjIyuoLswB/yFMx4j0sHsbLWXlcneJ2ZvLyxUyzvvqL2wsPeYI7Q5mJ0diGkCABAWrFapsjJCmZlmzZljUWamWZWVERrC4lgAADAIVtxgxHKEMTE5OeooKOgtQOz4M3rFCkIbAAB8rLHRoKwssxoaDE7dX2pqjEpMtKuiok0JCezMBwDAUwQ3GNEOZmfLOmtWv21QXbm56k5PZ3uUl1it0qZNESoujlJzs0FxcXbl5XUqI4O9xQAQzqxWKSvLrNra/j8MWlqMqq3tub26upWfFwAAeIjgBiPeYOEMoY138EkqAGAwmzZFqKHB4HJMfb1BVVURyswMTMteAABGOmrcABhU309S+4Y2kuOTVJOysszUMACAMFVcHNXv58PhWluNKiqK8tOMAAAIPQQ3AAa1eXOk25+kAgDCT3Oz658RQx0HAAD6I7gBMKgHHjDzSSoAYFBxce5tlXV3HAAA6I/gBsCg+CQVAOBKXl6nYmNtLsdYLDYtWdLppxkBABB6CG4ADIpPUgEEK6tVqqyMUGamWXPmWJSZaVZlZQQ1t/wsI6NbiYmufwYkJdmVnk5hYgAAPEVhCgCDuu66Nr33XoTL7VJ8kgrA3+h2FzxMJqmiok1ZWWbV1xvU2nro38NisSkpqeffg1bgoclq7eksVlwcpeZmg+Li7MrL61RGRjf/5gDgRQQ3AAZ1/vldSky0q7Z28DF8kgpguIZy8de3293herrd9dxeXd3KhaOfJCTYVV3dqqqqCBUVHfo3XLKkU+npXMCHKgJUAPAfghsAg+KTVAC+NtSLv02bItzudpeZSajsLyaTlJnZzdc8TBCgAoB/UeMGgEuOT1JLS9uVltatadOsSkvrVmlpu6qrW/k0DYDH+l78Hb4ls+fiz6SsLLNT3Zri4ii63QEBNpQAFUBoiY+P17x583z2+PPmzVN8fLzPHn+4fP36B0NwA+CIHJ+kbtzYptdfb9XGjW3KzGT5O4Dh8eTij2538ATFrL2LABXosXv3bsXHxys+Pl6XXXbZgGO2bNmi+Ph43XDDDcN6Lm8EGmvXru2d7/vvvz+sx/Kn9evXKz4+XuvXrx/w9nnz5mncuHF+npV/EdwAAICA8OTij253GKrGRoPmzrVo0aIYbd0aoffeM2nr1ggtWhSjuXMtamwk5BsqAlSgv6qqKr3++uuBnoZL69evl8HQ8325du3aAM9mYPfff7/+8Y9/BHoaQYfgBgAABIQnF395eZ2KjbW5HE+3Ozh4sh0PR0aACn8y1tUN6XggTJo0SUajUXfccUegpzKonTt36o033lBWVpYmTZqkJ554Ql1dXYGeVj9JSUk64YQTAj2NoENwg4AZCW/CAADf8eTiLyOjW4mJru9Htzs4UIvFNwhQ4S+jystlmT1bkSUlTscjS0pkmT1bo8rLAzQzZ1OnTtVll12mbdu26bnnnnP7fvX19VqyZIlSU1N19NFH6+STT9aSJUvU0NDgNC4+Pr53NY9jq1N8fLxyc3Pdfi7HCpvLL79cl156qfbt26eNGzcOOv7//b//pzPOOEMTJkzQtGnTdPvtt6ujo2PAsY5tXJ2dnbrrrrs0ffp0HXPMMfr2t7+tV199VZLU0tKiZcuWKTU1VRMmTNB3v/td/fvf/x70sRxyc3OVl5cnScrLy3N6/Yd/bY455hiNGzduwK/N9u3bdc011+jEE0/U0UcfrenTp+sXv/iF9u3bN+zX7w/8lEJAjCovV0xOjjoKCtTV55sqsqRE0StWqL2sTAezswM4QwCAr+Xldaqmxuhyu9ThF390u8NQDGU7Hh2x3OcIUGtrBx9DgIrhclwvGKxWxeTny263q+uaaxR1//2KWb5ckhSTkyNJQXHdsHz5cpWXl+vuu+/WvHnzZDrCD6Jdu3YpIyNDn3/+uTIyMpSamqra2lqtW7dOVVVVqqqqUkpKiiTplltu0aOPPqr6+nrdcsstvY8xY8YMt+bW3d2txx9/XEcddZTOO+88TZ06Vb/5zW+0du1azZ8/v9/4VatWaeXKlUpISNDVV1+tUaNGqby8XDt27HD5PD/+8Y/1/vvv63vf+57a29v15JNP6rLLLlNVVZVuuOEGdXZ26vvf/7727t2r8vJyZWdn65133tGYMWMGfcx58+apublZGzduVGZmZr/X3Pdrc9NNN8lkMslgMDiN27hxo3784x/LZDLpe9/7no477jjt2LFDZWVlevnll/XXv/7VKSzy9PX7EsEN/O7wN2FJ6srNVWRJSe/fg+lNGADgG55e/Dm63VVVRaioKErNzQbFxdm1ZEmn0tMpnI5DqMXiGwSo8LW+1wsOo5cvV+T99yvi4497jxms1qC5bkhKStK1116r4uJirV27Vj/60Y9cjr/hhhv0+eef6/e//73T2IcffljXX3+9brjhBlVUVEiS8vPzVV1drfr6euV/db00FJs2bVJjY6Ouu+46RUREaMqUKTr11FP1yiuvqKGhQYmJib1j6+rqtGrVKh177LF67bXXdPTRR0uSbr31Vp177rkun2ffvn16/fXXZTabJUnnnHOOrrnmGmVlZenss8/Wgw8+qIiInghixowZ+tWvfqW1a9f2rqgZyAUXXNAb3MybN09XXnml0+19vza/+MUvFBkZKaPx0HvSvn37tGjRIh111FHatGmTkpKSem976qmndO211+qee+7Rr3/962G/fl9iqxT8aqA34Zj8fMXOnNkb2kiH3oSDZfkjAMD7HBd/qalWWSzO2y4sFptSU62DXvzR7Q7uoBaL7zgC1NLSdqWldWvaNKvS0rpVWtqu6upWJSTwNYVnjHV1/a4XHPqGNg6O64ZgKLdw8803a8yYMbr33nt14MCBQcc1NDTob3/7m0466ST98Ic/dLrthz/8oU488US99tpr/bZMearvNimHyy+/XDabrV+npieffFLd3d1avHhxb2ghSWPGjNHNN9/s8nlWrFjRG9pIUlZWlkaNGqXm5mbdfffdvaGNJP3gBz+Q1LOFyZcee+wx7d+/X7fffrtTaCNJF198sWbOnKnyPtecw3n9vsSKG/iNqzdh4+7d/Y453oSts2bJ9tUyQQBAaGH1DHzJk+14cJ8jQGWbGbzJlpKijoICpw91j6SjoCAorhfGjh2r66+/XnfddZdKSkp00003DTju3XfflSTNmTOnt8uTg8FgUFpamnbs2KHt27c7rYbxxP/+9z+99NJLOuGEE/SNb3yj93h2drby8/O1fv16LVu2rHcejiAlLS2t32OdccYZLp/r61//utPfTSaTjj76aLW1tfULTY455hhJ0meffTb0FzUE//znP3v/rBsg3Ovs7NTevXu1d+9ejR8/fliv35cIbuA3I/lNGADgO1z8wVeoxQKMTI4amO5cN7QXFjrVzAy03NxclZWVafXq1frxj3884JiWlhZJclrR0VdCQoIkaf/+/cOez2OPPSar1arLLrvM6Xh8fLwyMjJUUVGhv/3tb/r2t7/t9JxHHXXUoPMazEC1akwm04DHHatvDh486N4L8dCXX34pSSorK3M5rq2tTePHjx/W6/cltkrBr7pyc9VeWOjW2GB7EwYAACPLcLbjAQisrtxc2ZKTXY6xJScH3fVCTEyMbr31Vu3fv1+//e1vBxwTGxsrSfr8888HvN1x3DFuONatWydJuvvuu506MsXHx/fW0HFspZIOhS9ffPFFv8dqbGwc9nz8zfE13Lp1q5qamgb9b9KkSZKC9/UT3MDvRuqbMAAAGHmoxQKMTJElJQOWU+jLuHt3v1bhweCqq67SCSecoAcffHDAOjWOjkdbt26V3e78HmS32/X3v//daZyk3i5V1gHKTgzm9ddf165duzR58mQtXLhwwP/Gjh2rDRs2qKmpSZI0ffr03rkdzjEvfzvSa3d1+7e+9S1J0rZt29x6rmB8/RLBDQJgJL8JAwCAkYdi1sDI0rfb7JHE5OcH3XWDyWTSbbfdps7OTq1atarf7UlJSTrzzDNVW1vrtNpF6ln9Ultbq7POOsupvs3YsWMlSZ988onb83A89s0336w//vGPA/53xRVXqKOjQ3/5y18kSZdccolMJpPWrFnjtCJo//79+s1vfuP+F8GLHK/9008/HfLtV155pWJjY3X33XerdoB9swcOHHAKdYLx9UvUuIGfDfVNWBIrbwAAAIAwYayrU/SKFUO6T/SKFepOTw+q2pgXXnihTj31VP3jH/8Y8Pb77rtPGRkZWrp0qTZt2qSTTjpJH3zwgV544QUdddRRuu+++5zGn3XWWaqoqNCPfvQjffe731V0dLROPvlkpaenD/j4+/fv13PPPSeLxaL58+cPOs8rr7xSa9as0dq1a/XTn/5UKSkpWrZsmQoLCzVnzhzNnz9fEREReu655zRt2jTt3LnT46+Jp0499VTFxMSopKRELS0tvfVnbrjhBkmHvjY5OTk6//zzFRMT0/u1Oeqoo/Tggw/qRz/6kebOnavzzjtPU6dOVWdnpz7++GNt3bpVp556qp5++mlJCsrXL7HiBn7k6ZtwMLT2AwAAAOB7tpQUtZeVyT7Akrjur+qQ9GU3mdReVhZUoY3DHXfcMehtU6dO1SuvvKIFCxboX//6l/7whz/oX//6lxYsWKCXX35ZX/va15zG//CHP9TSpUvV2Nio3/72t7rzzjv17LPPDvr4Tz/9tA4cOKCLLrrIqUX34aZNm6ZZs2appqZGb7/9tiTplltu0R/+8AeNGzdODz/8sCoqKnTRRRfp4YcfHsKr956xY8fqkUce0ZQpU/TnP/9Zd955p+68887e23/4wx/q5z//uT7//HPdd999/b426enp+tvf/qYFCxbo/fffV2lpqZ588knV19drwYIF+uUvf+n0fMH2+iXJ0NTUxMbeMNTR0aH6+nolJSUpOjrab887qrx8wJbgtuTkftunHG/CB7Oz/TY/9AjU+YGRgfMDrnB+wBXOD7jC+THyfP7554N2Rhquw68bDqxcqZZrrlHsn/+s0cuXS+J6AYfYbDZ1dXUpMjJSRmNwrE/x5vdHcLwihI2D2dn9EvT2wkK1vPOOU7cp3oQBAACA8NX3uqG9sFCdixZJkjoXLVJ7YSHXCwgr1LiB3zneXGNyctRRUNBbw8bxZ/SKFbwJAwAAAGHuYHa2rLNm9WyDstl6j3fl5gZdTRvAlwhuEBBOb8J98CYMAAAAwGGw6wKuFxBO2CqFgOFNGAAAAAAA1whuAAAAAAAAghTBDQAAAAAAQJAiuAEAAAAAAAhSBDcAAAAAAABBiuAGAAAAAOARu90e6CkAQcfb3xcENwAAAACAIYuOjlZHR0egpwEEnY6ODkVHR3vt8QhuAAAAAABDZjab1draqvb2dlbeAOpZadPe3q7W1laZzWavPW6E1x4JAAAAABA2jEajxo8fr7a2Nn3xxRc+fS6bzda7isFoZP0BnAXT+REdHa3x48d7dR4ENwAAAAAAjxiNRsXGxio2Ntanz9PR0aH9+/drwoQJXt2CgtAQ6ucHUSUAAAACzmqVKisjlJlp1pw5FmVmmlVZGSGrNdAzAwAgsFhxAwAAgIBqbDQoK8ushgaDWloOfa5YU2NUYqJdFRVtSkigfgYAIDyx4gYAAAABY7VKWVlm1daanEIbSWppMaq21qSsLDMrbwAAYYvgBgAAAAGzaVOEGhoMLsfU1xtUVcVCcQBAeCK4AQAAQMAUF0f1W2lzuNZWo4qKovw0IwAAggvBDQAAAAKmudn1apuhjgMAINQQ3AAAACBg4uLcKzrs7jgAAEINwQ0AAAACJi+vU7GxNpdjLBablizp9NOMEGi0hgcAZ1R5AwAAQMBkZHQrMdGu2trBxyQl2ZWe3u2/SSFg9u6N0FVXjdOnn5poDQ8AX2HFDQAAAALGZJIqKtqUmmqVxeK88sZisSk11aqKijaZTAGaIPzGapUWLz5BO3aMojU8APRBcAMAAICASkiwq7q6VaWl7UpL69a0aValpXWrtLRd1dWtrLAIE5s3R+p//4t0OYbW8ADCEe96AAAACDiTScrM7FZmJluiwtUDD5h14IDryxNHa3jOEwDhhBU3AAAAAAKO1vAAMDCCGwAAAAABR2t4ABgYwQ0AAACAgLvuujaNHu16CxSt4QGEI4IbAAAAAAF3/vldOuaYLpdjaA0PIBwR3AAAAAAIOJNJWrPmPzrxxIO0hgeAPghuAAAAAASF8eO79fLL+3T//e068USrRo+2afRouxIT7crP79D48dS3ARB+CG4AAAAABI19+4y6555offqpQQcOGHXggEEffGBSXl6M5s61qLGRrlIAwgvBDQAAAICgYLVKF18cr9pak1panC9VWlqMqq01KSvLLKs1QBMEgAAguAEAAAAQFLZsidcnn7guYlNfb1BVVYSfZgQAgUdwAwAAACAoPProBLW2ur5EaW01qqgoyk8zAoDAI7gBAAAAgoTVKlVWRigz06w5cyzKzDSrsjIibLYGtbS41zKquZk6NwDCB2sMAQAAgCDQ2GhQVpZZDQ0Gp/ouNTVGJSbaVVHRpoSE0O6qFBvrXkIVFxfaXwcA6CskV9zYbDaVlpbqrLPO0sSJE5WUlKTMzExt3LhxwPH79+/X8uXLNX36dCUkJGj69Olavny59u/f7+eZAwAAIBxZrVJWljnsi/IuWLBHFovN5RiLxaYlSzr9NCMACLyQC27sdrt+9KMfadmyZWppadFVV12l7Oxs7dy5UwsWLFBpaanT+La2Ns2bN09r1qzR1KlTtXjxYp100klas2aN5s2bp7a2tgC9EgAAAISLTZsi1NDgevtPOBTlPfPMJh13nOt0KinJrvT0bj/NCAACL+SCm+eee07PPfecTj/9dG3dulW//vWvtXr1ar3xxhtKSkrSbbfdpt27d/eOX716tWpqarR06VI988wzuuOOO/TUU09p2bJlqqmp0erVqwP4agAAABAOiouj+q20OVw4FOU1maSnnmpSaqq138obi8Wm1FSrKiraZHKvFA4AhISQC24qKyslSTfeeKNiYmJ6j48fP16LFy9WZ2en1q9fL6lndc7atWtlsVi0bNkyp8e58cYbFR8fr3Xr1sluZw8tAAAAfMfdYrvhUJT36KNtqq5uVWlpu9LSujVtmlVpad0qLW1XdXVryNf5AYDDhdxay8bGRklScnJyv9scx7Zs2SJJ2rVrlz777DOde+65MpvNTmOjo6OVlpamjRs3qq6uTlOmTPHxzAEAABCu3C22Gy5FeU0mKTOzW5mZbIkCgJBbcXPUUUdJktN2KAfHsQ8//FBST3AjSSkpKQM+liOscYwDAAAAfCEvr1OxsRTlBQD0F3Irbs477zw99dRT+t3vfqezzjpL0dHRkqR9+/appKREktTc3CxJvV2j4uLiBnys2NhYp3FH0tHRMay5+1NXV5fTn0BfnB9whfMDrnB+wJVwPj+sVmnz5kg98IBZzc0GxcXZdd11bTr//C6ZTNJ3viMde2ykduwY/HPVxESrvv3tVo2gXzmHJJzPDxwZ5wdcGYnnhyOrcEfIBTcXX3yx1q9fry1btigtLU3nnnuuuru7VVlZqaOPPlqSZPJRNbNPP/1U1hHWo3HPnj2BngKCGOcHXOH8gCucH3Al3M6PvXsjtHjxCfrf/yJ14MChX7/ffdeoY47p0po1/9H48d1avfqzAceNHt2tiRO79Pvf/0effhr6W4fC7fzA0HB+wJWRcn6YTKZBd/4MJOSCm4iIiN4VN0899ZQeeeQRjRkzRhdccIF+9rOf6ZRTTtH48eMlSWPGjJF0aAXO4VpaWpzGHcmxxx7rhVfgH11dXdqzZ48mTJigyMjIQE8HQYbzA66E2vlxpE/BMTShdn7Au8Lx/LBapauuGqe6ulH9bjtwIEJ1dRFaunSaXn55n5KSpC1bWvTSS5EqKTFr/36DxoyxKze3Teed1yWTaWIAXoH/hOP5AfdxfsCVUD8/Qi64kaSoqCjdeuutuvXWW52OO4oSf+Mb35B0qIZNXV3dgI/jqG3jbmHioSx1ChaRkZEjct7wD84PuBIK50djo0FZWWY1NBic2vC+916EEhPtqqhoo3uJh0Lh/IDvhNP5UVkZoU8/dZ0Cf/KJSa+9ZuktxJuVJWVltfcZYZQUHl8vKbzODwwd5wdcCdXzI+SKE7vy5JNPSpJ+8IMfSOoJZCZOnKg333xTbW1tTmM7Ojq0detWTZw4cUhLmAAAI4PVKmVlmVVba3IKbSSppcWo2lqTsrLMGmE7YAEEmeLiqH7vMYdrbTWqqCjKTzMCAIw0IRncDFRMuKKiQuvWrdM3v/lNXXjhhZIkg8GghQsXqrW1VatWrXIaf99996mpqUkLFy6UwWDwy7wBAP6zaVOEGhpcv7/X1xtUVRWSi1MB+Elzs3u/R7o7DgAQfkLyt9HzzjtPxx13nE444QRFR0frrbfeUnV1tY4//ng9/PDDTsWJly5dqhdeeEGrV6/Wu+++q1mzZmn79u168cUXNWPGDC1dujSArwQA4CtD+RTcsX0BAIYqLs697ZbujgMAhJ+QXHFz0UUXac+ePXr00Uf1wAMP6PPPP9fNN9+sv/3tb5o0aZLTWLPZrA0bNmjx4sXauXOnioqKVFtbq8WLF2vDhg0ym80BehUAAF/iU3AA/pCX16nYWJvLMRaLTUuWdPppRgCkni3TlZURysw0a84cizIzzaqsjGCLNIJSSK64yc/PV35+vtvj4+LitHLlSq1cudKHswIABBM+BQfgDxkZ3UpMtKu2dvAxSUl2paezsg/wl8GaE9TUGGlOgKAUkituAGCk41Mg3+NTcAD+YDJJFRVtSk21ymJxfs+xWGxKTbWqoqJNJteNpwB4Cc0JMBKF5IobABjJ+BTIP/gUHIC/JCTYVV3dqqqqCBUVRam52aC4OLuWLOlUeno3oQ3gR0NpTkCNOwQLghsACCJ9PwU6XM+nQD23V1e38ov+MDk+Bc/KMqu+3qDW1kMhmcViU1KSnU/BAXiNySRlZnZzIQgEGM0JMBKxVQoAgggtqv3L8Sl4aWm70tK6NW2aVWlp3SotbVd1dSsrmwAACDE0J8BIxG/+ABBE+BTI//gUHACA8EFzAoxErLgBgCDCp0AAAAC+Q3MCjEQENwAQRPgUCAAAwHcczQlcoTkBgg3BDQAEET4FAgAA8B1Hc4LUVKssFuffuSwWm1JTrTQnQNAhuAGAIMKnQAAAAEdmtUqVlRHKzDRrzhyLMjPNqqyMkNV65PvSnAAjDcWJASCI0KIaAOCK1drTgbC4OErNzQbFxdmVl9epjIxufjYgbDQ2GpSVZVZDg8GpqUNNjVGJiT2/Kx0pfKE5AUYSghsACDKOT4GqqiJUVHToF/MlSzqVns4v5gAQrrxxsQqMdFarlJVlVm1t/1+IWlqMqq3tub26upXfmRAyCG4AIAjxKRAAoC8uVoEemzdHqqHBdXfN+nqDqqoi+D0KIYMaNwCCwnD2KQMAEOo2bYpw+2IVCGUPPGB2WnE2kNZWo4qKovw0I8D3eGcHEHAs/QYAwLXi4ii3L1ZZZYBQ1tzsOsAc6jhgJGDFDYCA6rv0+/BfSHuWfpuUlWVm5Q0AIKxxsQr0iItz78M8d8cBIwHBDYCAYuk3AABHxsUq0OO669oUG2tzOcZisWnJkk4/zQgSZQ98jSshAAHF0m8AAI4sL69TNTVGlz8zuVhFODj//C4lJtpVWzv4mKQku9LT+b3RXyh74HusuAEQUCz9BgDgyDIyupWY6PrCh4tVhAOTSaqoaFNqqlUWi/PKG4vFptRUqyoq2uiu5ieUPfAPghsAAcXSbyD0WK3SCy9Eslwa8CIuVoFDEhLsqq5uVWlpu9LSujVtmlVpad0qLW1XdXUrqzv8iLIH/sFXD0BAsfQbCC2ff27UggUnq7ExWq2tLJcGvMlxsVpVFaGioig1NxsUF2fXkiWdSk/vJrRBWDGZpMzMbrbSBxhlD/yD4AZAQDmWfrNPGRj5rFbp4ovjVVc3qt9tPcule5ZTV1e3coEJeIiLVQDBhLIH/sFWKQABxdJvIHRs2hShTz5x/c3KcmkAAEIHZQ/8g+AGQMCxTxkIDcXFUU7bowbiWC4NAABGvry8Ttqz+wEfeQEICiz9BkY+lksDw2O19qxcKy4+VL8mL69TGRnUrwEQnCh74B8ENwAAwCtYLg14rrHRoKwssxoaDE6FPinsDSCYOcoeZGWZVV9vcFp5a7HYlJRkp+yBF7BVCgAAeEVeXme/WlWHY7l08LNapcrKCNq5+5HV2lO4u7bW1K87S09hb5Oyssz8GwAISpQ98D1W3AAAAK/IyOjWccdZtWPH4J8LsVw6uLHqIzA2bYpQQ4PrLYSOwt5sKQYQjCh74FusuAEAAF5hMklPPdWklJQDdIkbgVj1ETjFxVH9vuaHo7A3AIQvVtwAAACvOfpomx599H3t2DFFDzwQ21tgdcmSTqWnU2A1mLHqI3Ao7A0AcIXgBgAAeJXJJKWndykrqy3QU8EQDGXVB8GNd1HYGwDgClulAAAAwKqPAMrL61RsLIW9AQADI7gBAAAAqz68bCjduTIyupWY6PrrSmFvAAhfbJUCAACA8vI6VVNjdLldilUf7hlqdy6TSaqoaFNWlln19Qa1th66j8ViU1KSncLeABDGWHEDAAAAVn14iafduRIS7KqublVpabvS0ro1bZpVaWndKi1tV3V1K23YASCMseIGAAAArPrwkuF05zKZpMzMboo/AwCcsOIGAAAAklj14Q1D6c4FAIA7WHEDAACAXqz6GB66cwEAvI0VNwAAAICX0J0LAOBtBDcAAACAl+TldSo21uZyDN25AABDQXADAAAAeAnduQAA3kZwAwAAAHiJoztXaqpVFovzyhuLxabUVCvduQAAQ0JxYgAAAMCLHN25qqoiVFQUpeZmg+Li7FqypFPp6d2ENgCAISG4AQAAALyM7lwAAG9hqxQAAAAAAECQIrgBAAAAAAAIUgQ3AAAAAAAAQYrgBgAAAAAAIEgR3AAAAAAAAAQpghsAAAAAAIAgRXADAAAAAAAQpAhuAAAAAAAAghTBDQAAAAAAQJAiuAEAAAAAAAhSBDcAAAAAAABBiuAGAAAAAAAgSBHcAAAAAAAABCmCGwAAAAAAgCBFcAMAAAAAABCkCG4AAAAAAACCFMENAAAAAABAkCK4AQAAAAAACFIENwAAAAAAAEGK4AYAAAAAACBIEdwAAAAAAAAEKYIbAAAAAACAIEVwAwAA4AarVaqsjFBmpllz5liUmWlWZWWErNZAzwwAAISyiEBPAAAAINg1NhqUlWVWQ4NBLS2HPveqqTEqMdGuioo2JSTYAzhDAAAQqlhxAwAA4ILVKmVlmVVba3IKbSSppcWo2lqTsrLMrLwBAAA+QXADAADgwqZNEWpoMLgcU19vUFUVC5kBAID3EdwAAAC4UFwc1W+lzeFaW40qKory04wAAEA4IbgBAABwobnZ9WqboY4DAAAYCoIbAAAAF+Li3Cs67O44AACAoQjJ4MZut+u5557TBRdcoBNPPFETJ07Ut771LV1//fX673//22/8/v37tXz5ck2fPl0JCQmaPn26li9frv379/t/8gAAIKjk5XUqNtbmcozFYtOSJZ1+mhEAAAgnIRncrFixQldffbU+/PBDzZs3Tz/96U+VnJysRx55RGeeeabef//93rFtbW2aN2+e1qxZo6lTp2rx4sU66aSTtGbNGs2bN09tbW0BfCUAACDQMjK6lZjoejVNUpJd6endfpoRAAAIJyHX/mDPnj0qKSnRpEmTVF1drTFjxvTetmbNGi1fvlzFxcUqLi6WJK1evVo1NTVaunSp7rzzzt6xK1eu1KpVq7R69WotX77c768DAAAEB5NJqqhoU1aWWfX1BrW2Hvrcy2KxKSnJroqKNplMAZwkAAAIWSG34ubjjz+WzWbT6aef7hTaSFJ6erok6YsvvpDUs6Vq7dq1slgsWrZsmdPYG2+8UfHx8Vq3bp3sdvasAwAQzhIS7KqublVpabvS0ro1bZpVaWndKi1tV3V1qxIS+F0BAAD4RsituJkyZYoiIyP1xhtvqKWlRbGxsb23bd68WZJ05plnSpJ27dqlzz77TOeee67MZrPT40RHRystLU0bN25UXV2dpkyZ4r8XAQAAgo7JJGVmdiszky1RAADAf0IuuBk3bpxuu+023XbbbTrttNP0ve99TxaLRe+//75effVV/ehHP9J1110nqSe4kaSUlJQBH8sR1uzatcut4Kajo8NLr8L3urq6nP4E+uL8gCucH3CF8wOucH7AFc4PuML5AVdG4vkRHR3t9tiQC24k6Wc/+5mOOeYY3XDDDfrTn/7Ue/y0007TpZdeqlGjRklSb9eouLi4AR/HsVrH3e5Sn376qaxW63Cm7nd79uwJ9BQQxDg/4ArnB1zh/IArnB9whfMDrnB+wJWRcn6YTKZBF5AMJCSDm1//+tdatWqVbr31Vl1++eWKj49XTU2NfvnLX+rCCy/UQw89pO9///tef95jjz3W64/pK11dXdqzZ48mTJigyMjIQE8HQYbzA65wfsAVzg+4wvkBVzg/4ArnB1wJ9fMj5IKb1157Tffcc48WL16sm266qff46aefrieeeEKzZs3S8uXL9f3vf7+3eHFzc/OAj9XS0iJJ/YocD2YoS52CRWRk5IicN/yD8wOucH7AFc4PuML5AVc4P+AK5wdcCdXzI+S6Sh1egLivo446SieffLIaGhq0d+/e3ro1dXV1Az6WowYOhYkB3zAO8r032HEAAAAACDchF9w4ihE5Wn4fznE8MjJSU6ZM0cSJE/Xmm2+qra3NaVxHR4e2bt2qiRMnDmnvGQD3jCovl2X2bEWWlDgdjywpkWX2bI0qLw/QzAAAAAAgeIRccHP66adLktasWdNvC9Sjjz6quro6zZo1S7GxsTIYDFq4cKFaW1u1atUqp7H33XefmpqatHDhQhkMBr/NHwgHo8rLFZOTI4PVqpj8/N7wJrKkRDH5+T3Hc3IIbwAAAACEvZCrcTN//nw99NBDqq6u1imnnKLvfe97io+P1/bt2/XKK68oKipKhYWFveOXLl2qF154QatXr9a7776rWbNmafv27XrxxRc1Y8YMLV26NICvBgg9fUMbh5j8fEXdf7+Mu3f3HnOEN5J0MDvb7/MEAAAAgGAQcituTCaTnn76ad1555067rjj9PTTT6ukpEQ7duzQJZdcoldeeUVnnHFG73iz2awNGzZo8eLF2rlzp4qKilRbW6vFixdrw4YNMpvNAXw1QGgx1tX1C216b+sT2jg4whtq3gAAAAAIVyG34kaSoqKitHTpUrdXy8TFxWnlypVauXKlj2cGhDdbSoo6CgoUk5/v9n06Cgpko84UAOAIrFZp06YIFRdHqbnZoLg4u/LyOpWR0S2TKdCzAwDAcyEZ3AAIXl25uZLkVnjTXljYOx4AgME0NhqUlWVWQ4NBLS2HFpTX1BiVmGhXRUWbEhLsAZwhAACeC7mtUsBgaD0dPLpyc2VLTnY5xpacTGgDADgiq1XKyjKrttbkFNpIUkuLUbW1JmVlmTXALl0AAEYEghuEBVpPB5fIkpIBa9r0Zdy9u9+/FwAAh9u0KUINDa47gNbXG1RVxUJzAMDIRHCDkEfr6eDi+Lq7o++/FwAEitUqVVZGKDPTrDlzLMrMNKuyMoIVHEGiuDiq30qbw7W2GlVUFOWnGQEA4F189ICQRuvp4GKsq1P0ihVDuk/0ihXqTk+nQDGAgKB2SvBrbna92mao4wAACDasuEHIovV08LGlpKi9rEz2Adp7DFTzxm4yqb2sjNAGQEBQO2VkiItzLzhzdxwAAMGG4AYhy9F6eihoPe17B7Oz+4U37YWFannnHbUXFvYec4Q2rIACECjUThkZ8vI6FRtrcznGYrFpyZJOP80IAADvIrhBSOvKzXUKA1yh9bT/9A1v+n7dHf9ehDYAggG1U0aGjIxuJSa6Xk2TlGRXenq3n2YEAIB3Edwg5NF6OjgdzM5W67Zt/b7uXbm5at22jdAGQMBRO2VkMJmkioo2paZaZbE4r7yxWGxKTbWqoqJNA+zSBQBgRCC4Qcij9XTwGmxbGtvVAAQDaqeMHAkJdlVXt6q0tF1pad2aNs2qtLRulZa2q7q6lQLSAIARjU3ZCGlDbT0tiZU3AABJPbVTamqMLrdLUTsleJhMUmZmtzIz2RIFYOSwWntqqhUXR6m52aC4OLvy8jqVkdHNSkH0IrhByKL1NABgOBy1U2prBx9D7RQAgKcaGw3KyjKrocHg9CFBTY1RiYl2VVS0sWIQktgqhRBG62kAwHBQOwUA4CtWq5SVZVZtranfys6WFqNqa03KyjLLag3QBBFUCG4Q0mg9DQAYDmqnAAB8YdOmCDU0uC5uX19vUFUVm2TAVimEAUcYE5OTo46CAqfW01LP9ihCGwDAYKidAgDwtuLiKJc11CSptdWooqIofv6A4Abh4WB2tqyzZvXbBtWVm0tNGwAAAAB+1dzserXNUMchtLFVCmGD1tMAAAA4nNUqVVZGKDPTrDlzLMrMNKuyMoLaIvCpuDj3ttq6Ow6hjRU3AAAAAMISXX0QKHl5naqpMbrcLmWx2LRkSacfZ4VgxYobIAwZ6+qGdBwAACDU0NUHgZSR0a3ERNehYFKSXenp1LcBwQ0QdkaVl8sye7YiS0qcjkeWlMgye7ZGlZcHaGYAAAD+Q1cfBJLJJFVUtCk11SqLxeZ0m8ViU2qqVRUVberTHBdhjOAGCCOjyssVk5Mjg9WqmPz83vAmsqREMfn5PcdzchT17LOBnSgAAICPDaWrD+ALCQl2VVe3qrS0XWlp3Zo2zaq0tG6VlrarurqVbXroRXwMhIm+oY1DTH6+ou6/X8bdu3uPGaxWxeXlaexdd0k/+UkgpgoAAOBzdPVBMDCZpMzMblp+wyVW3ABhwFhX1y+06b2tT2jjYLBalXL77TJ99JE/pocRjm4coYN/SwDhhK4+AEYKVtwAYcCWkqKOggLF5Oe7fZ/6pUsVNXmyRvlwXhj56MYROvi3BBBu6OoDYKRgxQ0QJrpyc9VeWOjW2P13363GK67w8Yww0tGNI3TwbwkgHNHVB8BIQXCDkEOr68F15ebKlpzscowtOVntOTl+mhFGss2bI+nGESLorAIgHNHVB8BIQXCDkEKra9ciS0oGrGnTl3H3bsWUlflpRhjJHnjATDeOEEFnFQDeMtJqZdHVB8BIwEdnCBmHt7qWelaYOFpdS1LMVytJDmZnB2yegdL363AkY267TQlffiktW+bjWWEkoxtH6ODfEoA3jNRaWXT1ARDsWHGDkDBYq+vYmTOdwgqD1aqYnJywW3ljrKtT9IoVQ7pP0urVdJWCS3TjCB38WwIYLmplAYDvENxgxPOk1XVMTk5Y1byxpaSovaxM9gE2aQ9U88ZuMqnurrtknTzZH9PDCHXddW2KjbW5HEM3jpEhL6+Tf0sAw0KtLADwHYIbjHiOVtdD0VFQIFtKio9mFJwOZmf3C2/aCwvV8s47Tt2m7CaTmouL9eX55wdimhhBzj+/i24cIYLOKgCGi1pZAOA7BDcICUNpdd1eWKiu3Fwfzyg49Q1v+n4dHF8/u8mk9rIydc6fH9iJYkSgG0fo4N8SwHBRKwsAfIe1iggZXbm5irr/fpddk2zJyWEb2jgczM6WddasfiuOunJz1Z2e3nO8oyNAs8NI4+jGUVUVoaKiKDU3GxQXZ9eSJZ1KT+/mQn8E4d8SwHBQKwsAfIfgBiHD3VbXkSUlYR/eDLZNLNy2j8E76MYROvi3BOCpvLxO1dQYXW6XolYWAHiGrVIICUNpdR2Tn6/IkhIfzwgAACB8UCsLAHyH4AYjnietrqNXrAirrlIAAAC+RK0sAPAdghuMeJ60um4vK2NbEAAAgBc5amWVlrYrLa1b06ZZlZbWrdLSdlVXtyohYeAVOVarVFkZofnzx+qKK07W/PljVVkZIavVzy8AAIIUNW4QEg5mZ0uSYnJyZPjqp7yja1LfbVSO0MYxHgAAAN4z1FpZjY0GZWWZ1dBg+Ko+TqQ+/FB6770IJSbaVVHRNmjgAwDhghU3CBnutromtAEAAAg8q1XKyjKrttbUr6hxS4tRtbUmZWWZWXkDIOyx4gYhxa1W1wAAAAi4TZsi1NBgcDmmvt6gqqoIut0BCGusuEHIodU1AABA8CsujnLZPlySWluNKiqK8tOMACA4EdwAAAAA8LvmZterbYY6DgBCFcENAAAAAL+Li3Ov6LC74wAgVBHcAAAAAPC7vLxOxcbaXI6xWGxasqTTTzMCgOBEcAMAAADA7zIyupWY6Ho1TVKSXenpFCYGEN6GHdzs2rVLN910k2bPnq3jjjtO48ePd7p97dq1uvfee9Xa2jrcpwICzlhXN6TjAICelr+VlRHKzDRrzhyLMjPNqqyMoMUvEOZMJqmiok2pqVZZLM4rbywWm1JTraqoaJPJFKAJAkCQGFZw89RTT2nu3Ll66KGH9OGHH+rAgQOy251T86amJt1777166aWXhjVRINBGlZfLMnu2IktKnI5HlpTIMnu2RpWXB2hmABC8GhsNmjvXokWLYrR1a4Tee8+krVsjtGhRjObOtaixkaKjQDhLSLCrurpVpaXtOv30Ln3tawd0+uldKi1tV3V1qxISqG8DAB4HN++++65yc3PV1dWlnJwcbdiwQbNmzeo3LisrS3a7XRs3bhzOPIGAGlVerpicHBmsVsXk5/eGN5ElJYrJz+85npNDeAMAfVitUlaWWbW1pn4tf1tajKqtNSkry8zKGyDMmUxSZma3nn32Sz322Pt69tkvlZnZzUobAPhKhKd3/MMf/iCr1arCwkJdd911kqTo6Oh+4yZNmqSEhAS99957ns8SCKC+oY1DTH6+ou6/X8bdu3uPOcIbSTqYne33eQJAsNm0KUINDa5X1NTXG1RVFaHMzEM1LKzWnvsWF0epudmguDi78vI6lZHBhRwAAAg/Hgc3W7duVWxsbG9o48qxxx6rjz/+2NOnAgLGWFfXL7Tpva1PaOPgCG+ss2bJlpLijykCQNAqLo7qt9LmcK2tRhUVRfUGN42NBmVlmdXQYHC6b02NUYmJdlVUtLF1AgAAhBWPt0p98cUXmjx5sntPYjSqra3N06cCAsaWkqKOgoIh3aejoIDQBgAkNTe7V7/GMY6tVQAAAP15HNyMGTNGe/bscWvsRx991K/bFDBSdOXmqr2w0K2x7YWF6srN9fGMAGBkiItzb2WMY9xQtlYBAACEC4+DmxkzZmjPnj165513XI7btGmTvvzyS51yyimePhUQcF25ubIlJ7scY0tOJrQBgD7y8joVG2tzOcZisWnJkk5JQ9taBQAAwpfVKlVWRigz06w5cyyaP3+sXn01PmRX5Xoc3Fx++eWy2+26/vrr1djYOOCYDz74QDfeeKMMBoOuvPJKjycJBFpkScmANW36Mu7e3a9VOACEs4yMbiUmul51k5RkV3p6T32boW6tAgAA4aex0aC5cy1atChGW7dG6L33THrjjUj96lfH65xzxqmxMfR+T/A4uLn00kv17W9/W2+//bbOOOMM5eXl6ZNPPpEkPfDAA7rmmmv07W9/W5999pm+973vKT093WuTBvzJ0fLbHX1bhQNAuDOZpIqKNqWmWmWxOK+8sVhsSk21qqKirbdT1FC3VgEAgPDiqh7egQMR2rFjVEjWw/M4uDEYDFq3bp3mz5+vffv26dFHH1V9fb3sdrvy8/P1zDPPqKurS/Pnz9eDDz7ozTkDfmOsq1P0ihVDuk/0ihUy1tX5aEYAMLIkJNhVXd2q0tJ2paV1a9o0q9LSulVa2q7q6lanDlFD3VoFAADCS7jWwxvWq7FYLHrooYf0s5/9TM8++6y2b9+upqYmmc1mnXzyybrooot0+umne2uugN/ZUlLUXlY2YEtwW3Jyv+1TdpNJ7WVldJUCgD5MJikzs7u35fdgHFuramsHH9N3axUAAAgvQ6mHd6TfO0YSj4ObjRs3SpLOO+88ffOb39Q3v/lNr00KCCYHs7MlySm8cXSP6ruNyhHaOMYDAIbGsbUqK8us+nqDWlsP/WJmsdiUlGR32loFAADCS7jWw/M4uLnyyiuVmJiompoab84HCEp9w5uOgoLe7lGOP6NXrCC0AQAvcGytqqqKUFFRlJqbDYqLs2vJkk6lp3cT2gAAEMbCtR6ex8HNuHHjNGHCBG/OBQhqB7OzZZ01q982qK7cXHWnp7M9CgC8xN2tVQAAILzk5XWqpsbocrtUKNbD87g48de//nX997//ld0eWkkW4Mpg4UwohjZR9fUDHqfwMgAAAIBAcNTDcyUU6+F5HNwsWrRIe/fuVVlZmTfnAyAIRD37rKZfcoliDvv+jiwpkWX2bI0qLw/QzAAAAACEK0c9vNRUqywW506Uo0d366STDoZkPTyPt0qlp6fr7rvv1m233aadO3fqyiuv1IknnqiYmBhvzg+An40qL1dMXp4MVqvG3Hab2iMi+hVijsnJkSRq+gAAAADwq4Hq4cXGWnXxxbt1xRVjZDZHB3qKXjesGjcOf/rTn/SnP/3J5XiDwaC9e/d6+nQA/GBUeXm/1ucx+fmKuv9+p9bnBquV8AYAAABAQBxeD6+jo0P19c0ymcYEeGa+4fFWKbvdPqT/bDbbkR8UQMAY6+r6hTa9t/UJbRwc4Q01bwAAAADAdzxecfPOO+94cx4AAsyWkqKOgoLe7VDu6CgoCMnCzAAAAAAQLDwObiZNmuTNeQAIAl25uZLkVnjTXljYOx4AAAAA4Bseb5UCEJq6cnPVfYRg1pacTGgDAAAAAH7g8Yqbvrq7u/Xvf/9b//nPf9Ta2iqLxaITTzxR3/jGN2QKtT5cQIiLLClRxMcfuxxj3L1bkSUlhDcAAIwgVqu0aVOEiot7urDExdmVl9epjIzukGudCwChZNjBzf3336/f/e53+vzzz/vddvTRR+umm27ST3/60+E+DQA/6Nvy+0gc4whvAAAIfo2NBmVlmdXQYFBLy6FF9zU1RiUm2lVR0aaEBHsAZwgAGMywtkr97Gc/0/Lly9XY2Cij0ajjjjtOp5xyio477jgZjUY1Njbq1ltv1c9+9jNvzReAjxjr6hS9YsWQ7hO9YgVdpQAACHJWq5SVZVZtrckptJGklhajamtNysoya4DGkgCAIOBxcPP8889r3bp1GjVqlG6++WZ9+OGHqqmp0Ysvvqiamhp9+OGH+sUvfqHIyEitX79eGzZs8Oa8XVq/fr3i4+Nd/vf973/f6T779+/X8uXLNX36dCUkJGj69Olavny59u/f77d5A4FkS0lRe1mZ7AOslbYlJ/c7ZjeZ1F5WRlcpAACC3KZNEWpoMLgcU19vUFWVV6ooAAC8zON354cfflgGg0HFxcW6+OKL+90eHx+v5cuX68QTT9S1116rhx9+WBdccMGwJuuuGTNm6JZbbhnwtueee061tbU699xze4+1tbVp3rx5qqmp0dlnn62LL75Y27dv15o1a7RlyxZt2rRJZrPZL3MHAulgdrYkKSYnR4avPnZzdI/qu43KEdo4xgMAgOBVXBzVb6XN4VpbjSoqilJmZrefZgUAcJfHwc3bb7+tiRMnDhja9PWDH/xAt99+u/797397+lRD9vWvf11f//rX+x3v6upSWVmZIiIidMUVV/QeX716tWpqarR06VLdeeedvcdXrlypVatWafXq1Vq+fLlf5g4E2sHsbHV1dSkuL08td9wh+1c1bBy1bKJXrCC0ARBUKLgKuNbc7Hq1zVDHAQD8y+OtUq2trTrmmGPcGnvMMceotbXV06fymg0bNmjfvn1KT09XQkKCJMlut2vt2rWyWCxatmyZ0/gbb7xR8fHxWrdunex2irUhfHTOn6/tTz6p9pwcp+Ndublq3baN0AZA0GhsNGjuXIsWLYrR1q0Reu89k7ZujdCiRTGaO9eixkYuROE/VqtUWRmhzEyz5syxKDPTrMrKiIDXjomLc+/3WHfHAZ4I1u8PYCTweMXN+PHj9dFHH8lqtbps+W21WlVXV6fx48d7+lRes3btWknS1Vdf3Xts165d+uyzz3Tuuef22w4VHR2ttLQ0bdy4UXV1dZoyZYpf5wsEUmdS0oDHqWkDIFj0Lbh6uJ6Cqz23V1e3svIGPhfMXZvy8jpVU2N0uV3KYrFpyZJOP84K4SSYvz+AkcDj4Oa0005TRUWFfvOb3wxaT0aS7rvvPjU1Nenss8/29Km84uOPP9Zrr72mY489Vuedd17v8V27dkmSUga5GHWENbt27TpicNPR0eGl2fpeV1eX059AX5wfcIXzA6748/x44YVI1dcfueDqhg02padzvgaDUH3/sFqlCy8cpx07Bg8RL7wwRi+/vC8gIeJ3viMde2ykduwYPLhJTLTq299uVSB/nQ3V8yPceev7g/MDrozE8yM6OtrtsR4HN3l5eaqoqNC9996rt99+W0uWLNG0adMUHx+vpqYmvf/++youLtYLL7wgo9GovLw8T5/KK9avXy+bzaYFCxY4rRBydI2Ki4sb8H6xsbFO41z59NNPZR1ha/327NkT6CkgiHF+wBVfnx9Wq7RlS7wefXSCWlpMio21asGCPTrzzCZWT4wA/nj/+MMfTlRr65ELrv7+9xE6+eRdPp8P3BdqP19efTVe9fWuV5fX1xv02GP79e1vN/tpVs5Wr/5MixefoP/9L1IHDhy6BBg9ulsTJ3bp97//jz79NDgKE4fa+RHuvP39wfkBV0bK+WEymQZdPDIQj4Obb33rW7r77rt12223qaqqSlVVVT0PGBGh7u6eN3273S6DwaC7775bp5xyiqdPNWw2m03r16+XwWDQVVdd5bPnOfbYY3322N7W1dWlPXv2aMKECYqMjAz0dBBkOD/gij/Oj88/N+rii+P1yScmpwvznTvNOu44q556qklHH23zyXNjePz5/tHR4d4nVZ2d0UoaZPsn/CtUf748/fRYpzBkIAcOROipp5J11VVf+mlWzpKSpC1bWvTSS5EqKTFr/36DxoyxKze3Teed1yWTaWJA5tVXqJ4f4c5b3x+cH3Al1M8Pj4MbqWfVzYwZM/Sb3/xGW7duldVq1cGDB3seOCJCc+bM0c0336y5c+d6ZbKeeuWVV9TQ0KBvf/vbOv74451uGzNmjCSpuXngdLelpcVpnCtDWeoULCIjI0fkvOEfnB9wxVfnh9UqXXqpZcAl1a2tRu3YYdSll46jbkmQ88f7x9ix7hUejo838F4WZELt50tLi3tvRi0tpoC/7qwsKSurvc8Ro6Tg+rcItfMj3Hn7+4PzA66E6vkxrOBGks466yydddZZOnDggOrq6tTa2iqLxaKUlBSNHj3aG3MctoGKEjs46tbU1dUNeF9HDRwKEwOAf2zaFKGGhiPXLamqilBmZnAs60dgUHAVwYKuTcDg+P4Ahs/jduCHGz16tKZPn67TTz9d06dPD5rQZt++fdq4caPGjh2rCy64oN/tU6ZM0cSJE/Xmm2+qra3N6baOjg5t3bpVEydOHNL+MwCA54qLo1xeiEs9K2+KiqL8NCMEq4yMbiUmuv5FPynJrvR0Aj74Vl5ep2JjXW/fHEkhIm2b4U2h9v0BBILHwY3dbtf+/fvV3t7uclx7e7v2798vuz0wCerjjz+urq4uXXrppYqK6v9LvsFg0MKFC9Xa2qpVq1Y53eboiLVw4UIZDO4txwYADE9zs3vvt+6OQ+gymaSKijalplplsThfFFgsNqWmWlVR0caWOvhcKIWIjY0GzZ1r0aJFMdq6NULvvWfS1q0RWrQoRnPnWtTYyHsvhiaUvj/QH0Gvf3gc3Kxbt07HH3+8SkpKXI4rKSnR8ccfr8cff9zTpxqWdevWSRp4m5TD0qVLNWPGDK1evVoXXXSR7rzzTl1yySVatWqVZsyYoaVLl/prugAQ9lhSjaFISLCrurpVpaXtSkvr1rRpVqWldau0tF3V1a1KSOA8ge+FSohotUpZWWbV1pr6rXzsadtsUlaWmQsyDEmofH+gP18HvYRCh3hc4+b555+XwWDQlVde6XLcggULVFBQoIqKCl1xxRWePp1H3nrrLb3//vs65ZRTNG3atEHHmc1mbdiwQffee6+ee+45VVdXa8KECVq8eLFuueUWmc1mP84aAMIbdUswVCaTlJnZTc0jBJQjRKyqilBRUZSamw2Ki7NryZJOpad3j4iLUmqMwVdC4fsDzvoGvYfrCXp7bve0mURjo0FZWWY1NBicfiesqTEqMdGuioq2sPpwxuPgpra2VhMmTNCECRNcjjvmmGN0zDHHqLa21tOn8tgpp5yipqYmt8bGxcVp5cqVWrlypW8nBQBwybGk2tWPDZZUAwhGIz1EHEqNsZH6GhE4I/37A858GfT6OhQaiTzeKtXY2KiJEye6NXbixIlqbGz09KkAAGGEJdUAEBjUGAPgLl82kxhKKBQuPA5uRo8erb1797o1du/evYqMjPT0qQAAYYa6JQDgf9QYA+AuXwa9dBjtz+OI6qSTTtKbb76p999/XyeffPKg49577z3t3r1bs2fP9vSpAABhiCXVAOBf1BgD4C5fBr2s/uvP4xU3F154oex2uxYvXjxoHZmmpibl5eXJYDDowgsv9PSpAAAAAPgYbZsBuCsvr1OxsTaXYzwNeln915/HK25+/OMf689//rPeffddnXbaafrxj3+sU089VXFxcWpubtabb76pRx55RHv27NGUKVN07bXXenPeAAAAALzIUWMsK8us+nqDWlsPfcZrsdiUlGSnxhgASb5tJsHqv/48Dm5iYmL0l7/8RZdddpk+/PBDrVq1qt8Yu92uE044QY8//rhiYmKGNVEAAAAAvkXbZgDu8GXQS4fR/oZVhjklJUVbtmzRI488oueee061tbVqaWlRbGysTj75ZGVlZWnhwoWKjo721nwBAAAA+BA1xgC4w1dBL6v/+ht2/6zo6Ghdd911uu6667wxHwAAAAAAMAL4Kuhl9Z+z8Gl8DgAAAAAARgRW/x3i9eDmrbfe0rZt23Tw4EFNmTJF5557rqKiwqe/OgAAAAAAgLe4Hdw0NDToiSeeUHx8vH7yk5/0u/3AgQO65pprtHnzZqfjiYmJWrdunb7+9a8Pf7YAAAAAAABhZPD+WofZtGmT7rnnHn300UcD3r5s2TJVVVXJbrfLYDDo6KOPliTV19frsssuU0tLi3dmDAAAAAAAECbcDm62bt0qSfrBD37Q77b//ve/evTRR2UwGHThhRfqo48+0o4dO/Tmm29q6tSp2rNnj9auXeu9WQMAAAAAAIQBt4ObDz74QGazWd/4xjf63fbss8/Kbrdr7NixKi4u1pgxYyRJU6dOVWFhoex2u6qqqrw3awAAAAAAgDDgdnDz+eefa/LkyQPetnXrVhkMBp1//vmKjY11uu3cc89VfHy8duzYMbyZAgAAAAAAhBm3g5umpiaZBmmW/s4770iSzjzzzAFvP+6449TU1DT02QGAHxnr6oZ0HAAAAAB8ze3gxmw263//+1+/45988okaGxslSTNnzhzwvqNGjdKoUaM8nCIA+N6o8nJZZs9WZEmJ0/HIkhJZZs/WqPLyAM0MAIbOapUqKyOUmWnWnDkWZWaaVVkZIas10DMDAABD5XY78KlTp+qtt97S22+/rVmzZvUef+mllyRJFotFJ5988oD3/d///qeEhIThzRQAfGRUeblicnJksFoVk58vSerKzVVkSUnv32NyciRJB7OzAzZPAHBHY6NBWVlmNTQY1NJy6DO6mhqjEhPtqqhoU0KCPYAzBAAAQ+H2ipvzzjtPdrtdt9xyiz7//HNJPd2k7rvvPhkMBqWnp8tgMPS736effqr//e9/Ou6447w3awDwkr6hjUNMfr5iZ87sDW0k9YQ6OTmsvAEQ1KxWKSvLrNpak1NoI0ktLUbV1pqUlWVm5Q0AACOI28HNT3/6U40fP17btm3TySefrJNOOknf/OY39fHHH8toNCovL2/A+z333HOSpDPOOMM7MwYALzHW1fULbXpv27273zFHeEPNGwDBatOmCDU09P8gra/6eoOqqtxedA0AAALM7eBm7Nix+stf/qIJEyaou7tbe/bskd1ul8lk0sqVK522TznY7XY99NBDMhgMOvvss705bwAYNltKijoKCoZ0n46CAtlSUnw0IwAYnuLiqH4rbQ7X2mpUUVGUn2YEAACGa0gft3zzm9/UW2+9pc2bN+ujjz5SbGyszjvvPB1//PEDjv/yyy917bXXymAw6NRTT/XGfAHAq7pycyXJaVvUYNoLC3vHA0Awam52vdpmqOMAAEDgDXmd7OjRozV//ny3xo4bN045XxX0BIBg1ZWbq6j77x9we5SDLTmZ0AZA0IuLc6/osLvjAMDfrNaebZ/FxVFqbjYoLs6uvLxOfec7gZ4ZEDhscAYQ9iJLSlyGNlJPzZvIkhLCGwBBLS+vUzU1RpfbpSwWm5Ys6fTjrADAPa664h17bKRWr/5MSUkBnCAQIG7XuAGAUNS35feRxOTnK7KkxMczAgDPZWR0KzHR9WqapCS70tO7/TQjAHDPkbri7dgxSosXnxAWXfGsVqmyMkKZmWbNmWNRZqZZlZURYfHaMTBW3AAIW8a6OkWvWDGk+0SvWKHu9HQKFAMISiaTVFHRpqwss+rrDWptPXTxY7HYlJRkV0VFm0ymAE4SAAbgTle8//0vUi+9FKmsLD9NKgBcrTpKTOx5D09IYLtruGHFDYCwZUtJUXtZmewDXMHYkpP7HbObTGovKyO0ARDUEhLsqq5uVWlpu9LSujVtmlVpad0qLW1XdXUrv/ADCErudMU7cCBCJSVmP83I/4606qi21qSsLDMrb8IQwQ0QQox1dUM6Dulgdna/8Ka9sFAt77yj9sLC3mOO0OZgdnYgpgkAQ2IySZmZ3dq4sU2vv96qjRvblJnZzUobAEHL3W53+/eHblc8d1Yd1dcbVFXFxplwQ3ADhIhR5eWyzJ7drwZLZEmJLLNna1R5eYBmFvz6hjd9W3535eaqvbCQ0AYAAMDH3O12N2ZM6K4adGfVUWurUUVFUX6aEYIFUR0QAkaVlysmJ0cGq7W30G5Xbq5T4d2YnBxJInwYxMHsbFlnzeq3DaorN5eaNgAAAD7mTle80aO7lZvbplBdf+DuqiN3xyF0hOYZD4SRvqGNQ0x+vmJnznTqlmSwWhWTk8PKGxcGC2cIbeAuukAAAOAZd7riTZzYpfPO6/LTjPzP3VVH7o5D6GDFDTCCGevq+oU2vbft3t3vmCO8GWhlCYDhoQsEAACeO1JXvMREq37/+//IZJoYwFn6ljurjiwWm5Ys6fTjrBAMPFpx09TUpH/961+qra2V3X7kX0Jramr0+uuve/JUAFywpaSoo6BgSPfpKCggtAG8jC4QAAAMn6uueH/96z6NH98d6Cn6lDurjpKS7EpPD+2vA/ob0oqb1tZW3XjjjXrmmWdk/eq3z3Hjxun6669XXl6eDIaB99otW7ZM//jHP7R3797hzxiAE0ch3b7bogbTt/AuAO8ZSheIzEx+2QIAYDCOrniH/7zs6AjQhPzoSKuOkpJ6VvDSITD8uL3ixmaz6ZJLLtFTTz2l7u5u2e122e127d27V7fffrsuvPBCffnll4Pe352VOQA805WbK1tysssxtuRkQhvAR+gCAQAAvMHVqqPq6la2XYcpt4ObRx99VG+88YZGjRql2267Ta+99po2b96sa6+9VkajUVu3btW8efP0+eef+3K+AAYQWVIyYE2bvoy7d/drFQ7AO+gCAQAAvMWx6mjjxja9/nqrNm5sU2ZmNyttwpjbW6WeeuopGQwG/f73v9cVV1zRe3z27Nm67LLLdNVVV6m2tlYXXHCBnn/+eSUkJPhkwgCc9W35fSR9W4UD8B66QAAAAMBX3F5xU1NTo7FjxzqFNg7f+ta39Ne//lUnnHCC/vOf/2jevHn67LPPvDpRAP0Z6+oUvWLFkO4TvWKFjHV1PpoREJ7y8joVG2tzOYYuEAAAAPCE28HN/v37leyihsZxxx2njRs3atq0afrwww91wQUX6NNPP/XKJAEMzJaSovayMtkHWDc5UM0bu8mk9rIyukoBXkYXCAAAAPiK28GNxWLR/v37XY4ZP368NmzYoJkzZ6qurk4XXHCBGhoahj1JAIM7mJ3dL7xpLyxUyzvvqL2wsPeYI7Q5mJ0diGkCIc3RBSI11SqLxXnljcViU2qqlS4QAAAA8Ijbwc3UqVP10UcfqampyeW4+Ph4PffcczrllFP00Ucf6YILLqBgMeBjfcObvi2/u3Jz1V5YSGgD+AFdIAAAAOALbhcnPvPMM/XPf/5Tzz//vBYuXOhy7JgxY/TMM8/oBz/4gbZt2yZJMhjopAH40sHsbFlnzeq3DaorN1fd6elsjwL8wNEFIjOTLVEAAADwDrdX3KSnp8tut+uBBx5wa3xsbKyeeeYZpaWleTw5AEMzWDhDaAMAAAAAI5PbK25mz56tP/zhD7Lb7WpqalJ8fPwR72M2m/X000+rpKREnZ100gAADI2xrm7A4HGw4wAAAECocXvFjcFg0MKFC3X11Ve7Fdo4REdH64YbbtCtt97qyfwAAGFqVHm5LLNnK7KkxOl4ZEmJLLNna1R5eYBmBgAAAPiP2ytuAADwl1Hl5YrJyZHBalVMfr6knnpNkSUlvX+PycmRJIpuAwAAIKQR3AAAgkrf0MYhJj9fUfffL+Pu3b3HDFYr4Q0AAABCnttbpQAA8DVjXV2/0Kb3tj6hjYMjvDHW1fljegAAAIDfEdwAAIKGLSVFHQUFQ7pPR0EBhYoBAAAQsghuAABBpSs3V+2FhW6NbS8sVFduro9nBAAAAAQOwQ0AIOh05ebKlpzscowtOZnQBgAAACGP4AYAEHQiS0oGrGnTl3H37n6twgEAAIBQQ3ADAAgqfVt+H0lMfj7hDQAAAEIawQ0AIGgY6+oUvWLFkO4TvWIFXaUAAAAQsiK88SDvvfeePvroI7W1tclmsw067oorrvDG0wEAQpQtJUXtZWUDtgS3JSf32z5lN5nUXlZGVykAAACErGEFNxUVFbrtttvU0NDg1niCGwDAkRzMzpYkp/DG0T2q7zYqR2jjGA8AAACEIo+Dm+eff17XXHONbDaboqKilJycrKOOOkoGg8Gb8wMAhKG+4U1HQUFv9yjHn9ErVhDaAAAAICx4HNz89re/ld1u1xVXXKF77rlHY8eO9ea8AABh7mB2tqyzZvXbBtWVm6vu9HS2RwEAACAseBzc7NixQ/Hx8SoqKpLRSI1jAID3DRbOENoAAAAgXHgc3MTGxioxMZHQBgAAAAAAwEc8Tl1OPfVU/fe//3XZRQoAAAAAAACe8zi4ufnmm3XgwAH97ne/8+Z8AAAAAAAA8BWPt0rNmjVLDz74oPLy8rRt2zb98Ic/VEpKikaPHj3ofZKSkjx9OgAAAAAAgLDjcXAjSdOnT9cpp5yizZs3a/PmzS7HGgwG7d27dzhPBwAAAAAAEFY83iq1Y8cOnXvuuXr11Vdlt9uP+B+1cAAAw2GsqxvScSCUWa1SZWWEMjPNmjPHosxMsyorI2S1BnpmAAB4Bz/rDvF4xc1dd92lffv2aebMmfrlL3+pWbNm6eijj/bm3AAAkCSNKi9XTE6OOgoK1JWb23s8sqRE0StWqL2sTAezswM4Q8B/GhsNysoyq6HBoJaWQ5/B1dQYlZhoV0VFmxIS7AGcIQAAw8PPOmceBzdvvPGGYmJiVF5ernHjxnlzTgAA9HKENgarVTH5+ZKkrtxcRZaU9P49JidHkghvEPKsVikry6zaWlO/21pajKqt7bm9urpVpv5DAAAIevys68/jrVIHDx7U1KlTCW0wIrDFAhiZ+oY2DjH5+YqdObM3tJHUE+rk5GhUeXkgpgn4zaZNEWpoMLgcU19vUFXVsMoYAgAQMPys68/j4ObEE0/Ul19+6c25AD4xqrxcltmzFVlS4nQ8sqREltmzudADgpSxrq5faNN72+7d/Y45whsCWYSy4uIopyXjA2ltNaqoKMpPMwIAwLv4Wdefx8HNddddp/r6+iN2kwIC6fAtFo7wxrHFgk/pgeBlS0lRR0HBkO7TUVAgW0qKj2YEBF5zs+tPIIc6DgCAYMPPuv48Dm4uvvhi3Xjjjbr22mtVVFSklpYWb84LGDa2WAAjX1durtoLC90a215Y6FS4GAhFcXHuFWJ0dxwAAMGGn3X9eRzczJw5U0899ZTa29t1++23Kzk5WVOnTtXMmTMH/G/WrFlenLZ7nn/+ec2fP1+TJ0/WMccco69//ev6yU9+ooaGBqdx+/fv1/LlyzV9+nQlJCRo+vTpWr58ufbv3+/3OcM72GIBhI6u3FzZkpNdjrElJxPaICzk5XUqNtbmcozFYtOSJZ1+mhEAAN7Fz7r+PK7m8/HHH/c79sUXXww63mDw3zImu92uG264QQ8//LAmT56sH/zgB7JYLPrss8/0+uuvq76+XomJiZKktrY2zZs3TzU1NTr77LN18cUXa/v27VqzZo22bNmiTZs2yWw2+23u8A7HFou+K2uOhC0WwCHGuroBvx+MdXXSscf6dS6RJSUDBq59GXfvVmRJCeENQl5GRrcSE+2qrR18TFKSXenp3f6bFAAAXsTPuv48Dm6ef/55b87Dqx544AE9/PDDysnJ0f/93//JdFiPsO7uQ//Aq1evVk1NjZYuXao777yz9/jKlSu1atUqrV69WsuXL/fb3OE9jgs4d8IbtlgAhzi2GXYUFDh9X0SWlCh6xQpZi4ulU07xy1z6tvw+kr6twoFQZTJJFRVtysoyq77eoNbWQ4unLRabkpLsqqhoC5v2qACA0MPPuv4MTU1NIbUxrL29XSeffLLi4uL0z3/+UxERg2dTdrtdJ598slpaWrRjxw6nlTUdHR066aSTNHr0aL333nt+XTHkDx0dHaqvr1dSUpKio6MDPR2fip050+Wn9bbkZLW8844fZxT8wun8gLPDa0M5Qs2+AYrdZFLdXXfJ8pOf+PT8MNbVyTJ79oBbHgdjN5nUum0bq+cCiPcP/7BapaqqCBUVRam52aC4OLuWLOlUenp3UP8iy/kBVzg/4ArnR/gZys+6UD8/Qq7x+SuvvKIvv/xSCxYskNVq1caNG7Vr1y7FxcXpO9/5jlL6/DK/a9cuffbZZzr33HP7bYeKjo5WWlqaNm7cqLq6Ok2ZMsXfLwVewBYLwH2DFfSOuv9+p+8jg9WqlNtvV/P48dLll/tsPraUFLWXlQ1Yr8qWnNzve9tuMqm9rIzQBmHBZJIyM7uVmRk+y8QBAOGFn3WHhFxw8+9//1uSFBERoblz52rnzp29txmNRi1evFgFX7WX3bVrlyQ5hTl9OcKaXbt2uRXcdHR0DGvu/tTV1eX0ZyiKKStTzG23uTc2P18Hu7vVnpPj41mNDOFwfsCZ6aOPNGaIBb3j8vK0d+ZMWSdP9tm8OjIz1VVcrLi8vN657b/7brXn5CimrExjvvoet5tMai4uVmdmpjSC3otDEe8fcIXzA65wfsAVzg+4MhLPj6GsDPJKcPPvf/9b77zzjvbt26eDBw8OOu6WW27xxtO55CiQXFRUpJkzZ+rll1/WCSecoHfffVfXX3+9ioqKNHnyZP3kJz/p7RoVFxc34GPFxsZKktvdpT799FNZh7CkPxjs2bMn0FPwiaj6ek2/444h3Sf2jju0e9o0dSYl+WZSI1Conh8YQESEDixdqkn33ef2XeqXLlVjRIRUX+/DiUk65RSNvesupdx+e89zZmT0PGdGhhK+/FJJq1er7q679OUpp/h+LkHCapW2bInXo49OUEuLSbGxVi1YsEdnntkUNNtkeP+AK5wfcIXzA65wfsCVkXJ+mEymQReQDGRYwc3bb7+txYsX64MPPnA5zm63y2Aw+CW4sdl62oZFRkZq/fr1mjhxoiQpLS1NjzzyiObMmaOioiL95Cc/8fpzH+vnTivD0dXVpT179mjChAmKjIwM9HS8LylJzYd9Su/QPWmSIg7riub4tD4hLc2fswxaIX9+YGDLlmn/2LG9q1hc+fjGG2W4/nol+ev8+MlPtPeccxQ1ebKcotVly7T3kktkmTxZFv/MJOA+/9yoiy+O1yefmJyK9e3cadZxx1n11FNNOvpo1y00fYn3D7jC+QFXOD/gCucHXAn182NY7cDnz5+v5uZmnX766dq1a5e++OILXXrppfryyy/1r3/9S1988YViYmJ04YUX9uvs5CtjxoyRJM2aNas3tHFITU3V8ccfr7q6OjU1NfWObW5uHvCxWlpanB7zSEZiEaTIyMgROW+3XH652iMj3Sq02l5WJkN2tkL0K+GxkD4/MCD7z34m24MPuqwN1T1pkhqvuEJJ/j4/UlM1aijHQ5DVKl16qUU7dvT/mdraatSOHUZdeuk4VVe3BnzlDe8fcIXzA65wfsAVzg+4Eqrnh/HIQwb2xz/+Uc3NzVq2bJleeOGF3how999/v5544gl98MEHWr16tSRp3759Kioq8s6Mj2Dq1KmSBt/+5Dje0dHRO+e6uroBxzpq4FCYeOQ6mJ2t9rKynnCmT8vvrtxctRcW9oY2B7OzAzxTIDi4U9A74uOPlfDYY36aEfratClCDQ2uuxzW1xtUVRVyJewAAADClsfBzSuvvKLRo0fr5z//+YC3m0wmXX311SouLtZf//pXlZSUeDzJoTjzzDMlSf/5z3/63Xbw4EHV1dXJbDbrqKOO0pQpUzRx4kS9+eabamtrcxrb0dGhrVu3auLEiUPae4bgczA7W63btvXrGtWVm6vWbdsIbYCv9F2JdiST7rtPMWVlPp4RDldcHKWWFtc/ultbjSoqivLTjAAAAOBrHgc3n3zyiZKSknrbaBsMPZ8AHl6c+KKLLtIxxxyjJ554YhjTdN/kyZN1zjnnqK6uTv/v//0/p9t+97vfqbm5WfPmzVNERIQMBoMWLlyo1tZWrVq1ymnsfffdp6amJi1cuLD3tWHkGqw9MG2DgR7GujpFr1gxpPvE3nGHjIOsWIRvNDe79/PI3XEAAAAIfh6vpR41apRGjx7d+3dHB6bGxkYdd9xxTmMnTJigDz/80NOnGrLf/va3Ov/88/Xzn/9clZWVmjp1qt5991397W9/U1JSku6+++7esUuXLtULL7yg1atX691339WsWbO0fft2vfjii5oxY4aWLl3qt3kDQKDYUlLUXlbmVBOq97bk5H7bpxwFvQ2En34VF2f36jgAAAAEP49X3EycOFGNjY29f09OTpYkbdu2zWlcR0eH6urqers9+cPkyZP1yiuvaMGCBXr77bf1wAMPqK6uTjk5OXr55Zc1YcKE3rFms1kbNmzQ4sWLtXPnThUVFam2tlaLFy/Whg0belcUAUCo61sTyqG9sFAt77yj9sLC3mN2k0l1d92lzvnzAzDL8JaX16nYWNc/Ty0Wm5Ys6fTTjAAAAOBrHq+4mT59up5//nkdOHBAo0eP1ne+8x2VlZXprrvu0rRp0zR16lS1t7fr5ptv1v79+3Xaaad5c95HlJiYqDVr1rg1Ni4uTitXrtTKlSt9PCsACG6Omk8xOTnqKChwKugtSdErVqi5uFhfnnJK2LTfDiYZGd1KTLSrtnbwMUlJdqWnd/tvUgAAAPApj1fcnH/++Tp48KD++te/SpIyMjI0Y8YMffTRRzrttNM0ZcoUJScn67HHHpPBYBi0iDEAILgcqaA3K20Cx2SSKiralJpqlcXivPLGYrEpNdWqioq2gLcCBwAAgPd4HNxccMEFevzxx5WamtrzQEajnnzySZ1zzjmy2+3at2+fDh48qHHjxukPf/iDMjMzvTZpAIBvUdA7eCUk2FVd3arS0nalpXVr2jSr0tK6VVrarurqViUkUN8GAAAglHi8VcpsNis9Pd3p2IQJE/T000+rsbFRu3fvVkxMjFJTU2Xioz8AALzGZJIyM7uVmcmWKAAAgFDncXDjSkJCghISEnzx0AAAAAAAAGHD461SDQ0NQxr/wgsvePpUAAAAAAAAYcnj4ObMM8/U888/f8RxXV1d+sUvfqErr7zS06cCAAAAAAAISx4HN01NTfrhD3+om266SZ2dnQOO2blzp84991w9+OCDioqK8niSAAAAAAAA4cjj4ObXv/61oqKi9NBDD+nss8/WBx984HT72rVrdfbZZ2v79u2aOnWqNm/ePOzJAgAAAAAAhBOPg5trr71Wf/3rX3XCCSeotrZW55xzjh566CHt379fP/nJT7R06VK1tbXpyiuv1KuvvqoZM2Z4c94AAAAAAAAhz+PgRpJOPvlkvfrqq7r66qvV3t6um266SdOnT9czzzwji8WiP/3pTyoqKtLo0aO9NV8AAAAAAICwMazgRpKio6P1+9//Xt/97ndlt9vV0tIio9GoJ554QtnZ2d6YIwAAAAAAQFgadnCzZ88ezZ8/Xy+99JIMBoPGjx8vq9WqhQsX0gIcAAAAAABgGIYV3Lz00ks688wz9be//U0JCQkqLy/Xtm3blJGRob179+rKK6/UsmXL1NXV5a35AgAAAEDYsVqlysoIZWaaNWeORZmZZlVWRshqDfTMAPiax8HNbbfdpssuu0yff/65zj33XFVXV+s73/mOxo4dq8cee0z/93//p8jISD344IM699xztXPnTm/OGwAAAADCQmOjQXPnWrRoUYy2bo3Qe++ZtHVrhBYtitHcuRY1NhoCPUUAPuRxcFNUVCSTyaS77rpLTz31lI466iin26+77jq9+OKL+trXvqbt27fr7LPPHvZkAQAAACCcWK1SVpZZtbUmtbQ4X761tBhVW2tSVpaZlTdACPM4uJk8ebI2b96sn/3sZ4OOmTFjhl577TVdddVVOnDggKdPBQAAAABhadOmCDU0uF5RU19vUFVVhJ9mBMDfPA5uXnvtNc2aNeuI42JiYvTHP/5Rf/7znz19KgAIGGNd3ZCOAwAAeFNxcVS/lTaHa201qqgoyk8zAuBvHgc3sbGxQxo/f/58T58KAAJiVHm5LLNnK7KkxOl4ZEmJLLNna1R5eYBmBgAAwkVzs3v1a9wdB2DkGXY7cHf897//1fbt2/3xVADgFaPKyxWTkyOD1aqY/Pze8CaypEQx+fk9x3NyCG8AAEA/3uwAFRdn9+o4ACOP2xshx40bp9NPP10bN27sd1teXp6+9rWv6YYbbhjwvrm5ufrHP/6hvXv3ej5TAPCTvqGNQ0x+vqLuv1/G3bt7jznCG0k6mJ3t93kCAIDg09hoUFaWWQ0NBqctTjU1RiUm2lVR0aaEBPdDlry8TtXUGF1ul7JYbFqypHNY8wYQvNxecWO322W3D/wG8+ijj2rz5s1HvD8ABDtjXV2/0Kb3tj6hjYMjvKHmDQAA8EUHqIyMbiUmur6WSkqyKz2925MpAxgB/LJVCgBGCltKijoKCoZ0n46CAtlSUnw0IwAAMFL4ogOUySRVVLQpNdUqi8XmdJvFYlNqqlUVFW0ymTyaMoARgOAGAA7TlZur9sJCt8a2FxaqKzfXxzMCAAAjga86QCUk2FVd3arS0nalpXVr2jSr0tK6VVrarurq1iFtvQIw8rgf9QJAGOnKze1X0+ZwtuRkQhsAANDLlx2gTCYpM7NbmZlsiQLCDStuAGAAkSUlLkMbqafmzeGtwgEAQPiiAxQAXyC4AYDDOFp+u6Nvq3AAABDe8vI6FRtrczmGDlAAhorgBgD6MNbVKXrFiiHdJ3rFCrpKAQAAOkAB8Ikh1bhpaGjQvffeO+Tb6uvrhz4zAAgAW0qK2svKBmwJbktO7rd9ym4yqb2sjK5SAACgtwNUVpZZ9fUGtbYe+pzcYrEpKclOBygAQzak4OaTTz7xKLix2+0yGIZegAsAAuFgdrYkOYU3ju5RfbdROUIbx3gAAABHB6iqqggVFUWpudmguDi7lizpVHp6N6ENgCFzO7hJS0sjfAEQNvqGNx0FBb3doxx/Rq9YQWgDAAAGRAcoAN7kdnBTWVnpy3kAQNA5mJ0t66xZ/bZBdeXmqjs9ne1RAAAAAHyO4sQA4MJg4QyhDQAAAAB/ILgBAAAAAAAIUgQ3AAAAAAAAQYrgBgAAAAAAIEgR3AAAAAAAAAQpghsAAAAAAIAgRXADAAAAAAAQpAhuAAAAAAAAghTBDQAAAAAAQJAiuAEAAAAAAAhSBDcAAAAAAABBiuAGAAAAAAAgSBHcAAAAAAAABCmCGwAAAAAAgCBFcAMAAAAAABCkCG4AAAAAAACCFMENAAAAAABAkCK4AQAAAAAACFIENwAAABgSq1WqrIxQZqZZc+ZYlJlpVmVlhKzWQM8MAIDQExHoCQAAAGDkaGw0KCvLrIYGg1paDn0GWFNjVGKiXRUVbUpIsAdwhgAAhBZW3AAAAMAtVquUlWVWba3JKbSRpJYWo2prTcrKMrPyBgAALyK4AQDgCIx1dUM6DoSqTZsi1NBgcDmmvt6gqioWdQMA4C0ENwAAuDCqvFyW2bMVWVLidDyypESW2bM1qrw8QDODA8Ga/xQXR/VbaXO41lajioqi/DQjAABCH8ENAIQpLnaPbFR5uWJycmSwWhWTn98b3kSWlCgmP7/neE4O4U0AEaz5V3Oz69U2Qx0HAACOjOAGAMIQF7tH1je0cYjJz1fszJmKyc/vPUZ4EzgEa/4XF+de0WF3xwEAgCMjuAGAMMPF7pEZ6+r6hTa9t+3e3e+Y42vGaiX/IVgLjLy8TsXG2lyOsVhsWrKk008zAgAg9BHcAEAY4WLXPbaUFHUUFAzpPh0FBbKlpPhoRuiLYC1wMjK6lZjoejVNUpJd6endfpoRAAChj+AGAMIEF7tD05Wbq/bCQrfGthcWqis318czggPBWuCYTFJFRZtSU62yWJxX3lgsNqWmWlVR0SaTKUATBAAgBBHcAECY4GJ36Lpyc2VLTnY5xpacTGgTAARrgZOQYFd1datKS9uVltatadOsSkvrVmlpu6qrW5WQQH0bAAC8KSLQEwAA+I/j4rXvtqjBcLHbU/dnoNVIfRl371ZkSUnYf60CoSs3V1H33+/y34hgzTdMJikzs1uZmWyJAgDA11hxAwBhhlUk7nEUa3ZH3yLP8J+hBGsAAAAjFcENAIQZLnaPzFhXp+gVK4Z0n+gVK8K2HlAgEKwBAIBwQXADAGGEi1332FJS1F5WJvsAFVYHWq1kN5nUXlYW1vWA/IlgDQAAhBOCGwAIE1zsDs3B7Ox+4U17YaFa3nnHqSiuI7Q5mJ0diGmGJYI1AAAQTghuACBMcLE7dH3Dm77Fmh0djQhtAodgDQAAhAuCGwAII1zsDt3B7Gy1btvWr1hzV26uWrdt42sUQARrAAAgHNAOHADCjOMiNiYnRx0FBU4Xu1LP9igudp0NtuoonFcjBYuD2dmyzprV79+iKzdX3enp/BsBAIARj+AGAMIQF7sIJQRrAAAglLFVCgDCFBe7GGkGK5QdrgW0AQBAeCC4AQAAQW9Uebkss2f3a1EfWVIiy+zZGlVeHqCZAQAA+BZbpQAAQFAbVV6umJwcGaxWxeTnS+rZ1hdZUtL795icHEmiNhMAAAg5IbniZsaMGYqPjx/wvxtuuKHf+P3792v58uWaPn26EhISNH36dC1fvlz79+8PwOwBYGQwffTRgMfZtgJv6hvaOMTk5yt25sze0EZST6iTk8PKGwAAEHJCdsXNmDFjlHtY61ZJ+sY3vuH097a2Ns2bN081NTU6++yzdfHFF2v79u1as2aNtmzZok2bNslsNvtr2gAwIozdvFnjb7/dqSuV1LNtha5U8BZjXV2/0Kb3tt27+x1zhDcDFd4GAAAYqUI2uImLi1N+n0/iBrN69WrV1NRo6dKluvPOO3uPr1y5UqtWrdLq1au1fPlyX07V74x1ddKxxw54nF90ARxJ1LPPKuH229m2Ap+zpaSoo6DAaWXNkXQUFPCzDAAAhJSQ3CrlLrvdrrVr18pisWjZsmVOt914442Kj4/XunXrZLfbAzRD73MUd4wpK3M6TnFHAO4YVV6uuLw8tq3Ab7pyc9VeWOjW2PbCQqcVYAAAAKEgZIObrq4uPfroo/rtb3+rP/3pT6qpqek3ZteuXfrss8902mmn9dsOFR0drbS0NH366aeqC5F6DX3rBIy57TYlPPaYJPV+Ss6FFgBXPN22Qs0bDFdXbq5syckux9iSkwltAABASArZrVJ79uzR4sWLnY6dd955euCBBzR+/HhJPcGNJKUMsqR6ypQpveMc/+9KR0fHcKbsU1HPPquYwz4ln3Tffep+8klF1Nf3HnNcaHV1dalz/vwAzBTBoKury+lPQJJ07LGy33GHxtx2m9t3abnjDrUfe6wUxO+P8C5fvH/ElJUNGA72Zdy9W4Y//lHtX23TQ3Di5wtc4fyAK5wfcGUknh/R0dFujw3J4Oaqq67SnDlzlJqaqsjISO3YsUP33nuvXnzxRV1xxRWqqqqSwWDo7RoVFxc34OPExsZKktvdpT799FNZB/gkOtCi6us1/bDQxqFvaONgsFoVl5en7QkJ6kxK8scUEaT27NkT6Ckg2GRkKOHLLzXpvvuOOPTjG29UY0aGNMD7DEKft94/Eh57TBPcON8kacxtt6npyy/VeMUVXnlu+A4/X+AK5wdc4fyAKyPl/DCZTIMuIBlISAY3t9xyi9Pfv/Wtb+mJJ57QvHnz9Pe//12bN29Wenq615/32AEK/gaFpCS1ePApeUJamg8nhWDW1dWlPXv2aMKECYqMjAz0dBBkuq6/Xp2PP66oTz8ddEz3pEmKWrZMRL/hx5vvH6aPPtL41auHdJ+k1as1+pJLZJ08eVjPDd/g5wtc4fyAK5wfcCXUz4+QDG4GYjQatWDBAv3973/Xm2++qfT0dI0ZM0aS1NzcPOB9WlpaJKl33JEMZamTv9l/9jO1R0S41ZmjvbBQ9txcBe+rgb9ERkYG9XmNwIgpK3MZ2khSxMcfa8xDD1FzJIx55f0jNVXtZWUD1layJSf32z5lN5nUXlamUampGjW8Z4aP8fMFrnB+wBXOD7gSqudHyBYnHoijts2BAwckHaphM1jxYUcNHHfq24wEFHcEMFyRJSVur96Lyc9XZEkJxYkxLAezs9VeVia7ydR7rL2wUC3vvOPUbcoR2tCCHgAAhJqwCm7eeustSdKkSZMk9QQyEydO1Jtvvqm2tjansR0dHdq6dasmTpw4pL1nwSyypMSt4o6RJSV+mhGAkcRYV6foFSuGdJ/oX/5Slm99i251GJa+4U3flt+OVuGENgAAIJSFXHDzwQcfqKmpqd/xv//97youLlZUVJQuvPBCSZLBYNDChQvV2tqqVatWOY2/77771NTUpIULF8pgMPhj6j7laPntDsen5ADQly0lpd/KBwf7AOPtRqMMNpsMNpticnIIbzAsB7Oz1bptW79VoV25uWrdto3QBgAAhKyQq3HzzDPP6A9/+IPOOussTZo0SVFRUaqtrdXLL78so9Go3/3ud0rq0ylp6dKleuGFF7R69Wq9++67mjVrlrZv364XX3xRM2bM0NKlSwP4arzDo0/JV6xQd3q6bCGy2giAdxzMzlZXV5fi+nSqsxsMMtidoxu7JIPN1vt3g9WqmK/aNHOBDU8N9jOJn1UAACCUhdyKmzPPPFMZGRnauXOnHn/8cT3wwAP64IMPlJ2drc2bN+vqq692Gm82m7VhwwYtXrxYO3fuVFFRkWpra7V48WJt2LBBZrM5QK/Ee1x9St791baxvhxLzvlFGMBAOufPV91dd8luMg0Y2kjSQOsUHeENNW8AAAAA94Xcipu5c+dq7ty5Q7pPXFycVq5cqZUrV/poVoHn+IS7b2eOj2+8UVHLlmnMQw/1bqOiTgAAd3x5/vnae845Mr/6qtvbMCWpo6CAUBgAAAAYgpALbjC4vuFNyx13qDEjQ0lSb72A6BUrCG0AuM06ebK6UlMlya3wpm9RWQAAQoXVKm3aFKHi4ig1NxsUF2dXXl6nMjK6NcCCdwAYMoKbMHMwO1vWWbPUfuyxUn197/Gu3Fxq2gDwSFdurqLuv99l1zpbcjKhDQAg5DQ2GpSVZVZDg0EtLYeqUNTUGJWYaFdFRZsSEgYq4Q8A7gu5Gjc4Moo7AvCmyJISl6GNJBl376ZbHQAgpFitUlaWWbW1JqfQRpJaWoyqrTUpK8usr6oUAIDHCG4AAB6LLClxu8ZNTH4+4Q0AIGRs2hShhoaByvEfUl9vUFUVmxwADA/BDQDAI6aPPlL0ihVDuk/0ihV0lQIAhITi4qh+K20O19pqVFFRlJ9mBCBUEdwAADxinTxZ7WVlsg9QedGWnNzvmKNrHdsyAQChoLnZ9WqboY4DgMEQ3AAAPHYwO7tfeNNeWKiWd95Re2Fh7zFHaEPXOgBAqIiLc6/osLvjAGAwBDcAgGHpG970bfndlZur9sJCQhsACCCrVaqsjFBmpllz5liUmWlWZWUEBXO9IC+vU7GxNpdjLBablizp9NOMAIQqKmUBAIbtYHa2rLNm9dsG1ZWbq+70dLZHAUAA0KratzIyupWYaFdt7eBjkpLsSk/v9t+kAIQkVtwAfQxWNJViqsCRDRbOENoAgP/Rqtr3TCapoqJNqalWWSzOK28sFptSU62qqGjTAKXgAGBICG6Ar4wqL5dl9ux+7YojS0pkmT1bo8rLAzQzAACAoaFVtX8kJNhVXd2q0tJ2paV1a9o0q9LSulVa2q7q6lZWNAHwCt6pAfWENjE5OTJYrYrJz5fUs8UjsqSk9+8xOTmSRJ0OwIuMdXUDrsgZ7DgAwD1DaVWdmclWnuEwmaTMzG6+jgB8hhU3CHt9QxuHmPx8xc6c2RvaSOoJdXJyWHkDeAmr3IDAYWtw6KNVNQCEDoIbhDVjXV2/0Kb3tt27+x1zhDf8YgsMz+Gr3BzhjWOVG0Ep4DuEpuGBVtUAEDoIbhDWbCkp6igoGNJ9OgoK2MIBDAOr3IDAITQNH7SqBoDQQXCDsNeVm6v2wkK3xrYXFqorN9fHMwJCF6vcgMAhNA0uVqtUWRmhzEyz5syxKDPTrMrKCK91eXK0qnaFVtUAMDIQ3ADqCW9syckux9iSkwltgGFilRsQGISmwaWx0aC5cy1atChGW7dG6L33TNq6NUKLFsVo7lyLGhuHX3eGVtUAEDoIbgD1LBEf6BfXvoy7d/erBwBg6FjlBvgfoWnwsFqlrCyzamtN/bo+tbQYVVtrUlaW2Ssrb2hVDQChgXbgCHt9W34fSd9W4QA815Wbq6j773cZmLLKDfAux/eTOz/zCE19Z9OmCDU0uF5RU19vUFVVhFfaS9OqGgBGPlbcIKwZ6+oUvWLFkO4TvWIFS8eBYWKVGxAYbA0OvOLiqH4rbQ7X2mpUUVGUn2YEAAh2BDcIa7aUFLWXlck+wAbvgX6xtZtMai8rY+k4MER9w86hrnIjvAG8h9A08Jqb3atf4+44AEDoI7hB2DuYnd0vvGkvLFTLO+841eFwhDYHs7MDMU1gxBpVXi7L7Nk9F4yscgMChtA0OMTFuVdXxt1xAIDQR3ADyDm86buv31FEldAG8Ezf9sMx+fmKqKrq+V4z9P8kmVVugO8QmgaPvLxOxcbaXI6xWGxasqTTTzMCAAQ7ghvgKwezs9W6bVu/ff1dublq3baN0AYYor6hjUNMfr6i77xTBrvzJ8l2o1Edv/oVq9wAH2FrcPDIyOhWYqLr1TRJSXalp1NMGMAhVqtUWRmhzEyz5syxKDPTrMrKCK90oEPwo6sU0Mdgv6DyiyswNMa6un6hTe9tA9TXMNhsisnJUeu2bWpXzyf9hDaAdzm+n/p+bzpWmfbdRkVo6lsmk1RR0aasLLPq6w1qbT30OarFYlNSkl0VFW0aIGMDEKYaGw3KyjKrocHgVNy8psaoxMSe94yEBLZXhjJW3AAAvM6WkqKOgoIh3aejoEC2lBRWuQE+xNbg4JCQYFd1datKS9uVltatadOsSkvrVmlpu6qrW7kAA9DLapWyssyqrTX160jX0mJUba1JWVlmVt6EOFbcAAB8wnFB6E4x1L4XkBKr3ABfOpidLeusWf2+z7pyc9Wdns73n5+YTFJmZrcyM9kSBQzEapU2bYpQcXGUmpsNio2N0Q9+0KIFCwI9M//atClCDQ2uu8zV1xtUVRXB+0kIY8UNAMBnunJzB6yf0ZctOblfbSkAvsXWYADBrLHRoLlzLVq0KEZbt0bovfdMeuONSP3qV8frnHPGqbHRdZARSoqLo/qttDlca6tRRUVRfpoRAoHgBgDgM5ElJQPWtOnLuHs3bYcBAIAk11uDDhyI0I4do8Jqa1Bzs3shlbvjMDIR3AAAfKJvsdMjicnPJ7wBAABD2hoUDuLi3Kt55e44jEwENwAArzPW1Sl6xYr/3969h0Vd5v8ff3EUBARSUUxFcHMlNS3TzMxvpeaBSsNq03Jr12VrIDOtPLDoVcrKVzdzLZHSda9MzWwT08wTpa2ibrltP5X1mLgkeU5BQBA5/P7gO7OMDAgKMx9mno/r8ko+n3uGe/AtMq/u+33X6TE+CQlyz8xsoBkBABqD0lJp40Zvjjx2YWwNshYXd0UBAWU1jvH3L9NLL12x04zgCK4RUwIA7KosIkKFixfbPBK8LCysyvYp80k29NcAANd17py7Ro++XWfP+lgdk86Rx66FrUHWhgwpUdu25Tp4sPox7dqVa/BgGhM7M1bcAAAaROVjh80Kk5KUt3evCpOSLNc4fhgAUFoqPfFEkDIzm1qFNhJHHrsatgZZ8/CQ1q4tUGRkqfz9rVfe+PuXKTKyVGvXFqjSj1twQgQ3AIAGUzm8qXzkd7HJpMKkJEIbAICkir4mP/1U8ztPV+pr4srYGlRVSEi50tPztWhRofr2LVGXLqXq27dEixYVKj09n5VoLoDvfHA67pmZNrdbVHcdQMO6Gh2t0h49qvz9KzaZVDJ4MH8vAQBKTm5SZaXNtcx9TYYNY0uIM2NrkG0eHtKwYSXUv4tixQ2cildqqvx79apyOo13Sor8e/WSV2qqg2YGuLbqwhlCGwCARF8T/FdNW4OaNi1R585X2RoEl8OKGxfncfy4FBlZ5XpjXJ3ilZpqaYRqPoK42GSyOpLYNyZGktiWAQAAYCD0NUFl5q1Bmzd7asGCJsrNdVNAQKmeeCJLo0Y1k5+fj6OnCNgVK25cWPCWLWrer59TrE6pHNqY+U6dqoDu3S2hjaSKUCcmplG9NgAAAGcXF3elyuqKa7laXxNXZ94atGFDgXbuzNdnn13U//xPLitt4JIIblxUk88+U8T06ZbVKebwxrw6pTEFHO6ZmTaPHJZU5chh6b/hjXtmpj2mBwAAgOsYMqREt95a85FRrtjXBAAkghuX5JWaqsC4OKdZnVIWEaGixMQ6PaYoMbHRbQUDAABwVh4e0qef5igi4jJHHgPANehx42JudHWKrRNhjMR8xHDl4Kk6lY8kBgAAgDG0bFmmjz46oMOHO+r99wOUm+umwMByvfTSFQ0eXEJoA8BlseLGxTjz6pRik0llYWE1jikLCyO0AQAAMCgPD2nw4GJLX5MNGwo0bBihDQDXRnDjgopNJl2aObNWY+2xOqW6XjN17UHjnZJic9WQ1XNmZVVpxgwAAADYU2mp9MUXnho2zE/33eevYcP89MUXnrKxKB4ACG5cVWFMjK60aVPjGHusTvFKTZV/r143fbJV5SO/r6dyM2YAAADAns6edVO/fv568UVf7drlqX//20O7dnnqxRd91a+fv86edXP0FBsVQjC4AoIbF+W7eLGanDxZ45iGXp1S+QjvmznZyj0zUz4JCXX63D4JCZwqBQAAALsqLZWGD/fTwYMeysuzfiuWl+eugwc9NHy4H6FDLRGCwVUQ3Lgg75QUNZs2rVZjG2p1SuXQpvLnupGTrcoiIlS4eLHKbWx+ttXzptzDQ4WLFzeKvj0AAABwHps2eSo7u+Yw4cQJN23ezBky10MIBldCcONijLA65UZPtqppDlejo6uEN4VJScrbu1eFSUmWa+bQ5mp09E2+CgAAjK2+esgBqD/JyU2qhAzXys9314IFTew0o8aLEAyuhODGxRhhdUpDnWxVObyp3FS52GRSYVISoQ0AwGXUVw85wNnZO+DMza3d1p3ajnNlhGBwJQQ3LuhqdLRyk5MdujrFHKbURl1OtroaHa38PXuqjC82mZS/Zw+hDQDYGas+7K++esgBzs4RAWdgYHm9jnNlhGBwJQQ3LurKiBHKnDHDoatTik0mm6t8KruRk62qW5lDTxsAsC9WfdhfffaQA5yZowLOuLgrCggoq3GMv3+ZXnrpSr1+XmdECAZXQnDjwi4+/LB+Tk932OoU75QUmz1tKmvok60AAA2jyWefserDzhqihxzgjBwZcA4ZUqK2bWsOEtq1K9fgwSX19jmdFSEYXAnBjYsrDQ+3eb2hV6eYf3CvjYY62QoA0DCCt2xRYFwcqz7srKF6yAHOxNEBp4eHtHZtgSIjS+Xvbx06+PuXKTKyVGvXFshGO0pcgxAMroTgBnZnhJOtAAANw+P4cUVMn86qDwdpqB5ygLMwQsAZElKu9PR8LVpUqL59S9SlS6n69i3RokWFSk/PV0gIW3tqgxAMroTgBnZnhJOtAAANozQ8XCfGj6/TY1j1Ub8aqocc4CyMEHB6eEjDhpVow4YC7dyZrw0bCjRsWAkhQx0RgsFVENzAISof3W1m75OtAAAN4+yoUbo0c2atxrLqo/7RQw64PgJO50EIBldAcAOHqRzeOOpkKwBAwyiMieFNkQPQQw6oHQJOAI0JwQ0c6mp0tPL37HHYyVYAgIbhu3gxb4rsjB5yxlbd15mvv/0RcAJobAhu4HDV9TWg3wEANE4hK1eq2bRptRrLm6L6Qw854/JKTZV/r15Vat07JUX+vXpxspodEXACaIwIbgAAQL3xOH5c7ebPr9NjeFNUf+ghZzxeqamW46crB5XmVR/mk9UIb+yDgBNAY0RwAwAA6k1peLgyZ8zgTZED0UPOOCqHNma+U6cqoHt3q606hDf2RcAJoLEhuAEAAPXq4sMPKzc5mTdFDkQPOcdzz8ysEtpY7tno/2QOb1h9Zh8EnAAaE4IbAABQ766MGMGbogZQlwa39JBzrLKICBUlJtbpMUWJifz52BEBJ4DGguAGAAA0CN4U1S8a3DY+5qCyNioHnLAfAk4AjQHBDQAAaDC8KaofNLhtvIpNJpv9nSorCwsjtAEAVIvgBgAAwMBocNu4eaek2OxpU5l7VlaVlVQAAJgR3AAAABgUDW4bN/OKqNqovJIK9leX/lEAYG8ENwAAAAZFg9vGyz0zUz4JCXV6jE9CAkGBA9A/CoDREdwAAAAYGA1uG6eyiAjLyWpV7tnoeWM+aY3Qzb7oHwWgMSC4AQAAMDga3DZOV6Ojq4Q3hUlJytu71yqMM4c2nLRmX/SPAtBYENwAAAAYHA1uG6/K4U3lFVHmlVSENo5B/ygAjQnBDQAAgIHR4Lbxuxodrfw9e6qsiCo2mZS/Zw+hjQPQPwpAY+ISwc38+fMVFBSkoKAg7dmzx+aYS5cuKT4+Xl27dlVISIi6du2q+Ph4Xbp0yc6zBQAAqECDW+dR3Rt+ggDHoX8UgMbC6YObw4cPa9asWfLz86t2TEFBgaKiorRw4ULddtttio2NVefOnbVw4UJFRUWpoKDAjjMGAACoQINboGHRPwpAY+DUwU1paalMJpO6du2qqKioasfNnz9f+/fv1/jx47VmzRq98cYb+vTTTzVp0iTt379f8+fPt+OsAQAA/osGt0DDoX8UgMbAqYObP//5z8rIyNCCBQvkYeP/VElSeXm5li1bJn9/f02aNMnq3sSJExUUFKTly5ervLzcHlMGAACogga3QP2jfxSAxsJpg5sDBw5o9uzZeu211xQZGVntuGPHjunUqVO65557qmyn8vHxUd++fXXy5EllslccAAA4EA1ugfpD/ygAjYmnoyfQEEpKShQbG6tOnTppwoQJNY49duyYJCmimr3gHTt2tIwz/746RUVFNzBbxyguLrb6L1AZ9YGaUB/1x+P4cZWGh9f6emNAfTSwNm0kWz9vVHfdYKgP1MSu9dGmjUqTkxUYF1flSPCS9u3l+eOPVtfKPTyUm5ysK43k75oz4vsHatIY68PHx6fWY50yuJk7d64yMjL05ZdfysvLq8ax5lOjAgMDbd4PCAiwGleTkydPqvSab/xGd+bMGUdPAQZGfaAm1MfNCd6yRRHTp+vE+PE6O2qU5XrIypVqN3++MmfM0MWHH3bgDG8O9YGaUB+oid3qo2dPBc+YoYjp0y3hzY8TJ+rsqFEKWblS7d9+W1JFaJM5Y4Yu9uwpnThhn7mhWnz/QE0aS314eHhUu3jEFqcLbvbv36+33npL48aNU48ePez6udu0aWPXz3cziouLdebMGbVq1Ure3t6Ong4MhvpATaiPm9fks88U+H9vFNq//baCgoNVGBMj38WL1ez/3ihETJ+u3ObNdWXECMdOto6oD9SE+kBNHFIfY8cqt3lzBcbFKe+NN9QkJkbtJGnSJF0KDlbAG28oNzlZ/iNGyN8+M0I1+P6Bmjh7fThdcGMymRQeHq4pU6bUanyzZs0kSbm5uTbv5+XlWY2rSV2WOhmFt7d3o5w37IP6QE2ojxvjlZoq32uW5jebNk3+f/mL1ckmbqWlCoyLU6G3d6PsXUJ9oCbUB2pi9/p4+mnl9+6t8ogIVf6s5ePGKT8qSm7XXIdj8f0DNXHW+nC64CYjI0OS1KpVK5v3Bw0aJElavny5HnnkEUvfmuqaD5t74Fyvvw0AANfjnpkp35iYKv0UJNk8jtattFS+MTEq7dFDZXVYTgsAqJvqvsfyvRd14Z6ZabNmqrsO1JbTBTdjxoyxeX3Xrl06duyYhg4dqhYtWqh9+/aSKgKZ0NBQffPNNyooKLA6WaqoqEi7du1SaGhonfafAQBgS1lEhIoSE2t9/KwkFSUm8sMeAAAG55WaKt+YGBUlJlqd/uedkiKfhAQVLl7cKFfQwhicLrh59913bV43mUw6duyYJk6cqF69elmuu7m5acyYMZozZ47mzJmjN99803Lv7bffVk5Ojn7/+9/Lzc2twecOAHB+5h/mahPeFCYlVTn6GQAAGIs5tHErLbX8+15sMsk7JcXysW9MjCQR3uCGOF1wcyPGjx+vjRs3av78+dq3b5969OihjIwMpaWlqVu3bho/fryjpwgAcCLFJpOavPeeze1RZmVhYYQ2AAAYXOXQxsx36tQq/86btz9LhDeoO3dHT8AI/Pz8tH79esXGxuro0aNasGCBDh48qNjYWK1fv95q+xQAADfLOyWlxtBGquh5452SYqcZAQCAurrR3nXu1fRXBarjMsFNSkqKcnJyrLZJVRYYGKhZs2YpIyND586dU0ZGhmbNmqXAwEA7zxQA4MwqL5u+Ht+pUwlvAAAwKHPvurqwd++66kIiwqPGxWWCGwAAHM09M1M+CQl1eoxPQgI/XAEAYFDFJpMKk5JqNdbeveu8UlPl36tXlf8J5J2SIv9eveSVmmq3ueDmENwAAGAnZRERKly8WOUeHlXvhYVVuVbu4aHCxYs5VQoAAAMrNpls/jtemb17113bMNkc3phX/pq3bRHeNA4ENwAA2NHV6Ogq4U1hUpLy9u61+j925tCGBoYAABib0XrXVdcwOaB7d6vt2oQ3jQfBDQAAdlY5vKm8bNq83JrQBgCAxsFovetomOycCG4AAHCAq9HRyt+zp8qy6WKTSfl79hDaAABgcEbsXdcYGiaj7ghuAABwkOp+SOKHJwAAjM+oveuM3DAZN4bgBgAAAACAG2DU3nVGbJiMG0dwAwAAAADADTJi7zqjNUzGzfF09AQAAAAAAGjMrkZHq7RHjyrboIpNJpUMHmzXbdB1bZgsiZU3BseKGwAAAAAAbpIRetcZsWEybh7BDQAAAAAATsCoDZNxcwhuAAAAAABwEkZtmIwbR48bAAAAAACciDmM8Y2JUVFiolXDZKliexShTeNBcAMAAAAAgJMxUsNk3By2SgEAAAAA4ISM0DAZN4/gBgAAAAAAwKAIbgAAAAAAAAyK4AYAAAAAAMCgCG4AAAAAAAAMiuAGAAAAAADAoAhuAAAAAAAADIrgBgAAAAAAwKAIbgAAqGfumZl1ug4AAABUh+AGAIB65JWaKv9eveSdkmJ13TslRf69eskrNdVBMwMAAEBj5OnoCQAA4Cy8UlPlGxMjt9JS+U6dKkkqNpnknZJi+dg3JkaSdDU62mHzBAAAQONBcAMAQD2oHNqY+U6dqibvvSf3rCzLNbfSUsIbAAAA1BpbpQAAuEnumZlVQhvLvUqhjZk5vKHnDQAAAK6H4AYAgJtUFhGhosTEOj2mKDFRZRERDTQjAAAAOAuCGwAA6kGxyaTCpKRajS1MSlKxydTAMwIAAIAzILgBAKCeFJtMKgsLq3FMWVgYoQ0AAABqjeAGAIB64p2SYrOnTWXuWVlVjgoHAAAAqkNwAwBAPah85Pf1+E6dSngDAACAWiG4AQDgJrlnZsonIaFOj/FJSOBUKQAAAFwXwQ0AADepLCJChYsXq9zDo+o9Gz1vyj08VLh4MadKAQAA4LoIbgAAqAdXo6OrhDeFSUnK27vX6rQpc2hzNTraEdMEAABAI+Pp6AkAAOAszGGMb0yMihITLadHmf/rk5BAaAMAAIA6IbgBAKAeXY2OVmmPHlW2QRWbTCoZPJjtUQAAAKgTtkoBAFDPqgtnCG0AAABQVwQ3AAAAAAAABkVwAwAAAAAAYFAENwAAAAAAAAZFcAMAAAAAAGBQBDcAAAAAAAAGRXADAAAAAABgUAQ3AAAAAAAABkVwAwAAAAAAYFAENwAAAAAAAAZFcAMAAAAAAGBQBDcAAAAAAAAGRXADAAAAAABgUAQ3AAAAAAAABkVwAwAAAAAAYFAENwAAAAAAAAZFcAMAAAAAAGBQBDcAAAAAAAAGRXADAAAAAABgUAQ3AAAAAAAABkVwAwAAAAAAYFAENwAAAAAAAAZFcAMAAAAAAGBQBDcAAAAAAAAGRXADAAAAAABgUAQ3AAAAAAAABkVwAwAAAAAAYFAENwAAAAAAAAZFcAMAAADYmXtmZp2uAwBcF8ENAAAAYEdeqany79VL3ikpVte9U1Lk36uXvFJTHTQzAIAReTp6AgAAAICr8EpNlW9MjNxKS+U7daokqdhkkndKiuVj35gYSdLV6GiHzRMAYBwENwAAAIAdVA5tzHynTlWT996Te1aW5ZpbaSnhDQDAgq1SAAAAQANzz8ysEtpY7lUKbczM4Q09bwAABDcAAABAAyuLiFBRYmKdHlOUmKiyiIgGmhEAoLFwuuAmJydHkyZN0qBBg9SpUyeFhIQoMjJSjz76qNauXavy8vIqj7l06ZLi4+PVtWtXhYSEqGvXroqPj9elS5cc8AoAAADgjIpNJhUmJdVqbGFSkopNpgaeEQCgMXC64ObChQtasWKFmjZtqqioKL300ksaOHCgDh06pOeee06vvPKK1fiCggJFRUVp4cKFuu222xQbG6vOnTtr4cKFioqKUkFBgWNeCAAAAJxOscmksrCwGseUhYUR2gAALJyuOXFYWJiysrLk6Wn90vLy8jRo0CAtXbpUL774oiIjIyVJ8+fP1/79+zV+/Hi9+eablvGzZs3SnDlzNH/+fMXHx9v1NQAAAMA5eaek2OxpU5l7Vpa8U1IIbwAAkpxwxY2Hh0eV0EaSAgIC9NBDD0mSMv+vyVt5ebmWLVsmf39/TZo0yWr8xIkTFRQUpOXLl9vcXgUAAADUReUjv6/Hd+pUeaekNPCMAACNgdMFN9UpKirS9u3b5ebmps6dO0uSjh07plOnTumee+6Rn5+f1XgfHx/17dtXJ0+etAQ9AAAAwI1wz8yUT0JCnR7jk5DAqVIAAOfbKmWWk5OjlJQUlZWV6fz580pLS1N2drYmT56sjh07SqoIbiQpoppu/ZXHmX9fk6KionqafcMrLi62+i9QGfWBmlAfqAn1gZq4dH20aaPS5GQFxsVVORK8pH17ef74o9W1cg8P5SYn60qbNlIj+hnzZrh0feC6qA/UpDHWh4+PT63HOm1wk5ubq9mzZ1s+9vLy0syZM/XSSy9ZrplPjQoMDLT5HAEBAVbjrufkyZMqveYfYqM7c+aMo6cAA6M+UBPqAzWhPlATl62Pnj0VPGOGIqZPt4Q3P06cqLOjRilk5Uq1f/ttSRWhTeaMGbrYs6d04oQjZ+wQLlsfqBXqAzVpLPXh4eFR7QISW5w2uAkLC1NOTo5KS0uVnZ2t1NRUzZw5U998840++OADm31wblabNm3q/TkbSnFxsc6cOaNWrVrJ29vb0dOBwVAfqAn1gZpQH6gJ9SFp7FjlNm+uwLg45b3xhprExKidJE2apEvBwQp44w3lJifLf8QI+Tt6rnZGfaAm1Adq4uz14bTBjZmHh4fCwsI0YcIEeXh4aPr06Vq6dKnGjh2rZs2aSapYnWNLXl6eJFnGXU9dljoZhbe3d6OcN+yD+kBNqA/UhPpATVy+Pp5+Wvm9e6s8IkKVvwrl48YpPypKbtdcdzUuXx+oEfWBmjhrfbhMc2JJevDBByVJ6enpkv7bw6a65sPmHji16W8DAAAA1FZZNUvkq7sOAHBdLhXcnD59WpIs26Q6duyo0NBQffPNNyooKLAaW1RUpF27dik0NLROe88AAAAAAADqi9MFN/v27bO59enixYuaMWOGJGngwIGSJDc3N40ZM0b5+fmaM2eO1fi3335bOTk5GjNmjNzc3Bp+4gAAAAAAANdwuh43H330kZYtW6Z+/fqpffv2atq0qU6cOKEtW7YoPz9fjz32mJ588knL+PHjx2vjxo2aP3++9u3bpx49eigjI0NpaWnq1q2bxo8f78BXAwAAAAAAXJnTBTfDhw/XpUuX9M9//lO7d+/W5cuXFRwcrD59+ujpp5/WyJEjrVbQ+Pn5af369Zo9e7bWrVun9PR0tWrVSrGxsZo8ebL8/Pwc+GoAAAAAAIArc7rg5t5779W9995bp8cEBgZq1qxZmjVrVgPNCgAAAAAAoO6crscNAAAAAACAsyC4AQAAAAAAMCiCGwAAAAAAAIMiuAEAAAAAADAoghsAAAAAAACDIrgBAAAAAAAwKIIbAAAAAAAAgyK4AQAAAAAAMCiCGwAAAAAAAIMiuAEAAAAAADAoghsAAAAAAACDIrgBAAAAAAAwKIIbAAAAAAAAgyK4cWEeHh6OngIMjPpATagP1IT6QE2oD9SE+kBNqA/UxJnrwy0nJ6fc0ZMAAAAAAABAVay4AQAAAAAAMCiCGwAAAAAAAIMiuAEAAAAAADAoghsAAAAAAACDIrgBAAAAAAAwKIIbAAAAAAAAgyK4AQAAAAAAMCiCGyeTk5OjSZMmadCgQerUqZNCQkIUGRmpRx99VGvXrlV5eXmVx1y6dEnx8fHq2rWrQkJC1LVrV8XHx+vSpUsOeAWwt/nz5ysoKEhBQUHas2ePzTHUiOvo1q2bpR6u/TVhwoQq46kN1/T5559rxIgRCg8PV+vWrXXHHXdo7Nixys7OthpHfbiWFStWVPv9w/zrscces3oMNeI6ysvLtW7dOj3yyCP65S9/qdDQUN1999165ZVX9J///KfKeGrDtZSVlWnRokXq37+/QkND1a5dOw0bNkwbNmywOZ76cE6rVq3SK6+8ogceeEAhISEKCgrSihUrqh1/I3Xwt7/9TQ899JDatGmjsLAwPfnkk/r+++8b4uXUK7ecnJyq7+TRaGVmZur+++/X3XffrYiICAUHB+vcuXPatGmTzp07p+eee07z58+3jC8oKNCQIUO0f/9+Pfjgg+revbsyMjL05Zdfqlu3btq0aZP8/Pwc+IrQkA4fPqz+/fvL09NTBQUFSktLU69evazGUCOupVu3bsrNzZXJZKpy784779SQIUMsH1Mbrqe8vFwTJkzQBx98oPDwcA0YMED+/v46deqUdu7cqcWLF+vee++VRH24on379umLL76weW/dunU6ePCg3nzzTY0fP14SNeJq/vCHPyg5OVmtW7fWsGHDFBAQoIyMDG3dulX+/v7avHmzbr/9dknUhqspLy/Xc889p3Xr1ik8PFwDBw5UcXGxNmzYoHPnzmnOnDn6/e9/bxlPfTivbt266cSJE2revLmaNm2qEydOKDk5Wc8880yVsTdSB3PnztXMmTPVtm1bDR8+XAUFBUpNTVVRUZFWr16t+++/314vtc48HT0B1K+wsDBlZWXJ09P6jzYvL0+DBg3S0qVL9eKLLyoyMlJSxWqL/fv3a/z48XrzzTct42fNmqU5c+Zo/vz5io+Pt+trgH2UlpbKZDKpa9eu6tixoz755BOb46gR1xMYGKipU6dedxy14Xref/99ffDBB4qJidH//u//ysPDw+p+SUmJ5ffUh+u54447dMcdd1S5XlxcrMWLF8vT01OjRo2yXKdGXMeZM2eUkpKi9u3bKz09Xc2aNbPcW7hwoeLj45WcnKzk5GRJ1IarWbdundatW6c+ffpozZo18vX1lSRNnz5dDzzwgKZNm6bBgwcrLCxMEvXhzN59911FRESoffv2mjdvntWf77XqWgfHjh1TUlKSfvGLX+irr75SYGCgJOmFF17QgAED9PLLL2vPnj1V3kcbBVulnIyHh4fNYgsICNBDDz0kqWJVjlSRbi9btkz+/v6aNGmS1fiJEycqKChIy5cvt7m9Co3fn//8Z2VkZGjBggVV3nyZUSOoDrXhegoLCzV79mx16NBBSUlJNr9vmP/9oT5Q2fr163XhwgUNHjxYISEhkqgRV/Pjjz+qrKxMffr0sQptJGnw4MGSpPPnz0uiNlyReaXexIkTLaGNJDVv3lyxsbG6cuWKZbsM9eHcHnjgAbVv3/66426kDlasWKGSkhK9+uqrltBGkiIjI/X000/r+PHj2r59e/29mHpGcOMiioqKtH37drm5ualz586SKlLHU6dO6Z577qmyjMzHx0d9+/bVyZMnLUEPnMeBAwc0e/Zsvfbaa5bVV7ZQI66puLhYH330kebOnaslS5Zo//79VcZQG65n27ZtunjxoqKiolRaWqp169Zp3rx5+utf/1rlz5n6QGXLli2TJP3617+2XKNGXEvHjh3l7e2tf/zjH8rLy7O6t2XLFkmybFGgNlzP2bNnJcmyoqYy87UdO3ZIoj5Q4UbqID09XZIsixkqM1/buXNnA8765hhzHRBuWk5OjlJSUlRWVqbz588rLS1N2dnZmjx5sjp27CipouAlKSIiwuZzVB5n/j0av5KSEsXGxqpTp042m81WRo24pjNnzig2Ntbq2sCBA/X++++refPmkqgNV2Ru3Ofp6al+/frp6NGjlnvu7u6KjY1VYmKiJOoD//Xjjz/q73//u9q0aaOBAwdarlMjruWWW27RtGnTNG3aNN1zzz0aOnSo/P39deDAAX399dd6/vnn9cILL0iiNlxRixYtJElZWVn65S9/aXUvKytLkvTDDz9Ioj5Q4Ubq4NixY/L391erVq1qHG9UBDdOKjc3V7Nnz7Z87OXlpZkzZ+qll16yXDN32668VKyygIAAq3FwDnPnzrU07vLy8qpxLDXiep599lndd999ioyMlLe3tw4fPqzZs2crLS1No0aN0ubNm+Xm5kZtuCDzNoYFCxaoe/fu2rp1qzp16qR9+/bplVde0YIFCxQeHq6xY8dSH7BYsWKFysrKNHr0aKvtddSI6xk3bpxat26tCRMmaMmSJZbr99xzj5566inLzyTUhusZOHCgPv30U82bN0/9+/eXj4+PJOnChQtKSUmRVPHeRqI+UOFG6uDSpUtq2bJlrccbDVulnFRYWJhycnL0888/a+/evYqPj9fMmTM1ZswYq+aRcC379+/XW2+9pXHjxqlHjx6Ong4MaPLkyerXr5+aN2+ugIAA3X333Vq1apXuvfdeffvtt5Yl7XA9ZWVlkiRvb2+tWLFCd911l/z9/dW3b18tXbpU7u7uWrBggYNnCSMpKyvTihUr5ObmpmeffdbR04GD/elPf1JsbKwmTJigf//73/rpp5+0adMmlZSU6NFHH9W6descPUU4yBNPPKH7779fu3fvVt++ffX6669rwoQJ6tOnj+UNdXX9GAFXQXDj5Dw8PBQWFqYJEyYoISFB69ev19KlSyXJ0hzOnGBfy7wH+domcmi8TCaTwsPDNWXKlFqNp0YgVWyDGT16tCTpm2++kURtuCLzn2WPHj0UGhpqdS8yMlIdOnTQ8ePHlZOTQ31AUkVfpOzsbPXv318dOnSwukeNuJa///3v+uMf/6iYmBi9+uqruvXWW+Xn56c+ffpo1apV8vX1tZz+Qm24Hk9PT3366aeaMmWK3N3dtXTpUn3++ecaNmyYPvzwQ0mybNWmPiDdWB00a9as2hU1jaFuCG5cyIMPPijpv42ZzHv5qmveZd7jx/5Q55GRkaEjR46oVatWCgoKsvxauXKlJGnQoEEKCgrS+vXrJVEj+C/zD0yXL1+WRG24ottuu01S9cuSzdeLioqoD0iy3ZTYjBpxLdc2IK6sRYsWuv3225Wdna2ff/6Z2nBRTZo00ZQpU/TPf/5TZ8+e1Q8//KA///nPOnnypCTpzjvvlMT3DlS4kTro2LGj8vPzdebMmVqNNxp63LiQ06dPS/rvca0dO3ZUaGiovvnmGxUUFFh15C4qKtKuXbsUGhpabdMnND5jxoyxeX3Xrl06duyYhg4dqhYtWliO4aNGYPbdd99JErXhwsxvuI4cOVLl3tWrV5WZmSk/Pz+1aNFCrVq1oj5c3IULF7RhwwYFBwfrkUceqXKf7yGupbi4WNJ/e2Vdy3zd29ub2oCVv/3tb5KkkSNHSuJ7ByrcSB3cd999+vbbb7V161aNGjXK6vm2bt1qGWNUrLhxMvv27bO5ZOzixYuaMWOGJFlOdXBzc9OYMWOUn5+vOXPmWI1/++23lZOTozFjxsjNza3hJw67ePfdd23+6t27tyRp4sSJevfdd3XHHXdIokZczaFDh5STk1Pl+u7du5WcnKwmTZro0UcflURtuKLw8HA99NBDyszMtCxdN5s3b55yc3MVFRUlT09P6gP6+OOPVVxcrKeeekpNmjSpcp8acS19+vSRJC1cuLDKz6kfffSRMjMz1aNHDwUEBFAbLsrWFpa1a9dq+fLluuuuu/j5A1ZupA6eeeYZeXp6au7cuVbfhw4ePKiPP/5Y4eHh6t+/v91eQ1255eTklDt6Eqg/U6ZM0bJly9SvXz+1b99eTZs21YkTJ7Rlyxbl5+frscce0wcffCB394rMrqCgQEOGDNH+/fv14IMPqkePHsrIyFBaWpq6deumTZs2WSWYcE4mk0krV65UWlqaevXqZXWPGnEdSUlJeuedd9S/f3+1b99eTZo00cGDB7V161a5u7tr3rx5VlseqA3Xc/z4cT388MM6d+6cBg8erNtuu0379u3T9u3b1a5dO3355ZeWYzapD9fWt29fHThwQDt37lSXLl1sjqFGXEdpaamGDx+u9PR0tWjRQkOHDlVQUJAyMjK0bds2NWnSRJ999pnuvfdeSdSGK+rdu7duvfVWderUST4+Pvruu++Unp6uDh06aN26dZYVvxL14cw+/PBD7d69W5J04MAB7d27V3369FF4eLgkKSoqyrKK80bq4K233lJiYqLatm2r4cOH6/Lly1q9erUKCwu1evVqghvYz+7du7Vs2TL985//1OnTp3X58mUFBwere/fuevrppzVy5MgqCbT56PB169bpzJkzatWqlR577DFNnjy52l4GcC41BTcSNeIq0tPTtWTJEu3du1fnzp1TUVGRQkJC1KdPH8XGxqpnz55VHkNtuJ7s7GzNmjVLX331lS5cuKBWrVpp6NChmjRpUpVjNqkP1/Tdd99pwIAB6tmzp7766qsax1IjruPKlSt67733lJqaqqNHj6q4uFghISG67777NGHCBN1+++1W46kN15KUlKTPP/9cJ06c0NWrVxUWFqZHH31UL7/8ss2GsdSHczK/J6nO5MmTNXXqVMvHN1IHn3zyiVJSUnTo0CF5eXmpd+/eio+P11133VXvr6c+EdwAAAAAAAAYFD1uAAAAAAAADIrgBgAAAAAAwKAIbgAAAAAAAAyK4AYAAAAAAMCgCG4AAAAAAAAMiuAGAAAAAADAoAhuAAAAAAAADIrgBgAAAAAAwKAIbgAAAAAAAAzK09ETAAAAxhcVFaWdO3fWamxOTk7DTsagDh06pJ07d+r777/X999/r8OHD6ukpESjRo1SSkrKTT33hQsXtHjxYqWlpeno0aMqKChQUFCQWrZsqc6dO6tv374aMmSI2rVrV0+vBgAAGAXBDQAAqLW2bduqbdu2jp6GIc2YMUMbNmyo9+f917/+paeeekrnz5+XJLVq1UodOnRQaWmpjh8/roMHD2rNmjXKycnR66+/Xu+fHwAAOBbBDQAAqLVnnnlGU6dOdfQ0DCk0NFRRUVG688471aNHD/3tb3/TqlWrbuo5CwoK9Mwzz+j8+fO6++67NWfOHN11112W+2VlZfr++++1evVqBQUF3eQrAAAARkRwAwAAUA/mzp1r9fHmzZtv+jm3bNmiU6dOycPDQ8uXL1fr1q2t7ru7u6tnz57q2bPnTX8uAABgTDQnBgAADeLSpUu68847FRQUpD/84Q82x/zlL39RUFCQwsLC9OOPP1qu5+fna9WqVRo7dqx69+6t9u3bq3Xr1urZs6dee+01ZWVl2Xy+HTt2KCgoSN26dZMkffrppxo0aJDatWuniIgIjR49WocOHbKM/3//7//p2Wef1W233abWrVvrf/7nf7Ru3bp6/CrcnOPHj0uSmjdvXiW0qa3Lly9r4cKFGjp0qDp06KCQkBB17dpVTz75pD788EOVlpZWecyePXv0m9/8RpGRkQoJCVFERISio6Or/dpkZWUpKCjIsupn8+bNevzxxxUREaGgoCCtX7/eMrasrEyrVq3S448/ro4dO6ply5aKjIzU2LFjtXfv3ht6jQAAODOCGwAA0CCaNWumJUuWyMvLSwsXLtSWLVus7h84cEAJCQmSpHfeeUft27e33EtPT9cLL7ygtWvXKj8/XxEREWrfvr1Onjypv/zlL+rfv7++//77Gj//jBkz9Lvf/U6nTp1Shw4ddPnyZW3YsEFDhw5VZmam1q9fr8GDBys9PV233nqrfHx8tHfvXj333HNas2ZN/X9BbkBAQIAk6ezZs8rMzKzz4zMzM9W/f3/Fx8dr9+7datasmbp06aKSkhJ9+eWXevnll5WXl2f1mOTkZD388MNas2aNCgsL1aVLFzVp0kRbt27Vr3/9a8XFxam8vLzaz/nuu+/qV7/6lfbu3asOHTpY9UTKy8tTdHS0XnjhBW3btk2enp6KjIxUQUGBVq9erQEDBuiTTz6p8+sEAMCZEdwAAIAGc9ddd2n69OkqLy9XbGysTp06JUkqLCzUb3/7WxUVFen555/X8OHDrR7XsWNHLV26VP/5z3904MABff311/r222915MgRvf7668rNzVVsbGy1AcKpU6e0aNEirVy5UhkZGdqxY4cyMjLUvXt3Xbx4Ua+++qpiY2P16quv6ocfftDXX3+tH374QaNGjVJ5ebmmTZtWYzhhL4MGDZKHh4ckacSIEfrggw+UnZ1dq8cWFhbqV7/6lX744Qd169ZNO3bs0L59+7Rt2zYdOnTIEpx5eXlZHrN9+3YlJCSovLxckyZN0tGjR7Vt2zYdPHhQixYtkre3t1asWFHjKVkzZsxQUlKSjh49qq1btyojI0MDBw6UJL388sv6+uuvdccdd2jbtm06fPiwtm/frv/85z9KSkpSWVmZxo0bp6NHj97EVw0AAOdCcAMAAGpt9uzZli0xtn6ZTKYqj3nppZc0YMAAnT9/Xr///e9VVlamKVOm6NChQ+rcubOSkpKqPOa2227T8OHD5e/vb3U9ICBAf/jDH9SnTx8dPHhQ//rXv2zOs6SkRJMmTdLQoUMt11q0aGHZsrVt2zb16dNHkyZNkqdnRcs/T09P/fGPf1STJk2UnZ2tjIyMG/461ZcOHTpo1qxZcnd3148//qhXXnlFXbt2VadOnfTkk09q3rx5lu1U1/rwww919OhRtWzZUp999pll+5hZaGioXnvtNfn5+VmuvfXWWyovL9fgwYMVHx9vFeo89dRTevnllyVJ8+bNU3Fxsc3PO2bMGJlMJkvgJEk+Pj767rvvtGbNGgUHB2vVqlW68847Lffd3d1lMpn0u9/9TleuXNHChQvr/sUCAMBJ0ZwYAADU2vWOA//FL35R5Zqbm5vee+899evXTzt27NCvfvUrpaWlycfHR0uWLJGvr6/N5yotLdWmTZv09ddfKysrS3l5eZZVMMeOHZMk7d27t9rGvM8991yVaz169Kjx/i233KKwsDAdOXJEx48frxJ2OMILL7yg3r17a8GCBdqyZYvy8vJ09uxZpaWlKS0tTTNmzNDo0aM1Z84cqxBm7dq1kqTnn39ezZs3v+7nKSgo0M6dOyVVhG22xMXFad68eTp37py+//573XPPPVXGPPvsszYf+9lnn0mShgwZotDQUJtjHnvsMS1atEjbt2+/7nwBAHAVBDcAAKDWbvQ48JYtW+q9995TdHS00tLSJEmJiYnq0qWLzfGnT5/WU089pX379tX4vBcuXLB5vXnz5goMDLQ5D7OIiAibj23RooWOHDmigoKCGj+3Pd15551asmSJSktLdeDAAe3du1c7duzQli1bdPHiRa1YsULnz5+3On784MGDkmQzXLElMzPT0qi4uj+X4OBghYaGKjs7W0eOHLH53L/85S9tPta8gik9PV1DhgyxOaaoqEiSdPLkyVrNGQAAV0BwAwAA7KJ79+665ZZb9PPPP8vf319PPPFEtWPj4uK0b98+dejQQdOmTVPv3r0VEhKiJk2aSKpYhbJq1SpdvXrV5uObNm1q87qbm1utx5SVldXqddmTh4eHunXrpm7duunZZ59Vbm6u4uLitH79em3evFl79uxRr169JMnSdNhWgGVLfn6+pIotY7fccku141q3bq3s7GzL+GtVXvVTWU5OjiTpxIkTOnHiRI1zKSwsrMWMAQBwDfS4AQAAdhEXF6eff/5Z7u7uys/P14QJE2yOO3PmjL766itJ0scff6yRI0eqXbt2ltBGki5evGiXORtdYGCgkpOT5e5e8SPdnj17LPfMJ1Ll5ubW6rnM/YRKSkqqXckkVayGqjy+tsyBTlJSknJycq77CwAAVCC4AQAADe7999/Xxo0bFRwcrM8//1zNmjXTmjVr9OGHH1YZm5WVJaliW07nzp2r3C8pKam2KbErCgwMVIsWLSTJagXS7bffLkn65ptvavU8ERERlkbNBw4csDkmJyfHcjJYdVuiqlPX+QAAgAoENwAAoEHt379f06dPlyS9++67uu+++zRv3jxJ0pQpU3T48GGr8eZmxXl5eTb7zHz00Uc6f/58A8/aGH7++efrbtk6evSozp07J6niGHUz8xHrS5curXEFjZmfn5/uu+8+SVJycrLNMQsXLlRpaalatmxpdSpUbTz++OOSpC+++KLaYAgAAFRFcAMAABrM5cuXNXbsWF25ckW/+93v9Mgjj0iSRo4cqWeeeUaXL1/Wb3/7W0tTWkmKjIxU8+bNVVJSotdee82q38maNWs0efJk+fj42P21OMLq1avVp08fpaSk6KeffrK6V15erq+++kqjR49WeXm52rZtqwEDBljujxkzRp06ddLZs2f1+OOPVzne/PTp05o7d65VOPbaa6/Jzc1NGzdu1Jw5c1RSUmI1l/nz50uSJk6caHVUeG3ce++9GjFihK5evaqRI0dq48aNllPCzLKysvTOO+/YXIkFAICrojkxAACotRUrVujvf/97jWMWLlxoObFp8uTJOnLkiG6//XYlJiZajZszZ46+/fZb/fvf/9a0adP0pz/9SVJFc9w33nhD48aN08qVK/XFF18oIiJCZ86c0alTpzRgwAA1b95cn3zyScO8yBu0evVqvf7665aPL1++LElKTU3V5s2bLdf/9Kc/aeTIkbV6Tjc3Nx05ckRTp07V1KlT1bp1a7Vu3VpXr17VTz/9ZOkFExISouXLl1sdre7r66uPP/5YTzzxhPbu3at+/fopLCxMzZs316lTp3T69GmVl5dr7Nixlsfcf//9SkxMVEJCgmbNmmX5szx9+rTlpKfRo0frxRdfvKGv0cKFC3XlyhVt3LhRo0aNUnBwsMLDw1VWVqaTJ0/q7NmzkirqBgAAVCC4AQAAtZadna3s7Owax5hPG1qzZo2WLVsmX19f/fWvf62ySsbPz09LlizRoEGDtHjxYj3wwAOKioqSVLFaJDg4WO+8847279+vo0ePKjw8XLGxsTKZTBo3blzDvMCbUFRUZHNL0pUrV3TlyhWrcbX1m9/8Rl26dNG2bdu0e/du/fTTTzp69KiuXr2qoKAg3X///Xr44Yf161//2ubpUREREdqxY4eWLFmidevW6ciRIzp9+rRCQkL08MMP69FHH7U0MTaLi4tT7969lZycrH/84x/av3+//P399eCDD+r555+3bMG6EU2bNtVHH32kTZs2acWKFfruu++UkZEhPz8/hYaGqn///ho6dKgGDRp0w58DAABn45aTk1N+/WEAAAAAAACwN3rcAAAAAAAAGBTBDQAAAAAAgEER3AAAAAAAABgUwQ0AAAAAAIBBEdwAAAAAAAAYFMENAAAAAACAQRHcAAAAAAAAGBTBDQAAAAAAgEER3AAAAAAAABgUwQ0AAAAAAIBBEdwAAAAAAAAYFMENAAAAAACAQRHcAAAAAAAAGBTBDQAAAAAAgEH9f2gQneG3WXbKAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1200x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "positive = data[data['admitted'].isin([1])]\n",
    "negative = data[data['admitted'].isin([0])]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(12,8))\n",
    "ax.scatter(positive['exam1'], positive['exam2'], s=50, c='b', marker='o', label='Admitted')\n",
    "ax.scatter(negative['exam1'], negative['exam2'], s=50, c='r', marker='x', label='Not Admitted')\n",
    "ax.legend()\n",
    "ax.set_xlabel('Exam 1 Score')\n",
    "ax.set_ylabel('Exam 2 Score')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2dp3JtvpjBYR"
   },
   "outputs": [],
   "source": [
    "def get_X(df):#读取特征\n",
    "#     \"\"\"\n",
    "#     use concat to add intersect feature to avoid side effect\n",
    "#     not efficient for big dataset though\n",
    "#     \"\"\"\n",
    "    ones = pd.DataFrame({'ones': np.ones(len(df))})#ones是m行1列的dataframe\n",
    "    data = pd.concat([ones, df], axis=1)  # 合并数据，根据列合并\n",
    "    return data.iloc[:, :-1].as_matrix()  # 这个操作返回 ndarray,不是矩阵\n",
    "\n",
    "\n",
    "def get_y(df):#读取标签\n",
    "#     '''assume the last column is the target'''\n",
    "    return np.array(df.iloc[:, -1])#df.iloc[:, -1]是指df的最后一列\n",
    "\n",
    "\n",
    "def normalize_feature(df):\n",
    "#     \"\"\"Applies function along input axis(default 0) of DataFrame.\"\"\"\n",
    "    return df.apply(lambda column: (column - column.mean()) / column.std())#特征缩放"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "mMuhs2-hjBYW",
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3, 1)\n",
      "(3, 1)\n"
     ]
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "\n",
    "def get_X(data):\n",
    "    # 这里假设 get_X 函数是从数据中提取特征的函数\n",
    "    # 示例：仅返回 col1 列作为特征\n",
    "    return data[['col1']]\n",
    "\n",
    "\n",
    "def get_y(data):\n",
    "    # 这里假设 get_y 函数是从数据中提取目标变量的函数\n",
    "    # 示例：仅返回 col2 列作为目标变量\n",
    "    return data[['col2']]\n",
    "\n",
    "\n",
    "data = pd.DataFrame({'col1': [1, 2, 3], 'col2': [4, 5, 6]})  # 示例数据\n",
    "X = get_X(data).to_numpy()\n",
    "print(X.shape)\n",
    "\n",
    "y = get_y(data).to_numpy()\n",
    "print(y.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8R0uQXSZjlH7"
   },
   "source": [
    "看起来在两类间，有一个清晰的决策边界。现在我们需要实现逻辑回归，那样就可以训练一个模型来预测结果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "R_GK_8JhjlH8"
   },
   "source": [
    "## Sigmoid 函数\n",
    "$g$ 代表一个常用的逻辑函数（logistic function）为S形函数（Sigmoid function），公式为： \\\\[g\\left( z \\right)=\\frac{1}{1+{{e}^{-z}}}\\\\] \n",
    "合起来，我们得到逻辑回归模型的假设函数： \n",
    "\t\\\\[{{h}_{\\theta }}\\left( x \\right)=\\frac{1}{1+{{e}^{-{{\\theta }^{T}}X}}}\\\\] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6yd5Ddl2jlH-"
   },
   "outputs": [],
   "source": [
    "def sigmoid(z):\n",
    "    return 1 / (1 + np.exp(-z))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "4SG4dpr3jlID"
   },
   "source": [
    "让我们做一个快速的检查，来确保它可以工作。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "FN4JMcM2jlIE"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA/MAAALMCAYAAACop+JvAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8fJSN1AAAACXBIWXMAAA9hAAAPYQGoP6dpAABqLUlEQVR4nO3dd3wUdeLG8WfTISGBAAm9IyKchXDSpQcBFZQS1BNsHPwUFbEBHlJOjXCe7U6Q0wgqSi+CIggeHUSJWCgqChJaCC2Fljq/P+YSCCmkbWZn9/N+vfa1352d2X0265o8zHdnHIZhGAIAAAAAALbhZXUAAAAAAABQPJR5AAAAAABshjIPAAAAAIDNUOYBAAAAALAZyjwAAAAAADZDmQcAAAAAwGYo8wAAAAAA2AxlHgAAAAAAm6HMAwAAAABgM5R5AIDHmTRpkhwOh7p06WJ1lHLncDjkcDi0fv16S7b/6aefNHjwYNWsWVM+Pj5yOBy68cYbS/RYdlPanx0AAJfzsToAAADwDAcOHFCHDh2UkpIiSQoNDZWvr6+qVatmcbLSeeONN5SYmKj+/ft7zD9MAACsR5kHAHicatWqqVmzZqpXr57VUcpds2bNJEkVK1Ys9+eeOXOmUlJS1KRJE61bt0516tQp9wzO8MYbb+jgwYNq0KBBoWXeyp89AMD9UOYBAB5n1KhRGjVqlNUxLPHzzz9b9tw//fSTJKlfv35uU+SLw8qfPQDA/fCdeQAAUC7Onz8vSQoKCrI4CQAA9keZBwDY3oIFC9S7d2+Fh4fL19dXlStXVtOmTXXHHXfo7bff1sWLF3OtX5QD4H366afq3r27KleurKCgIN1www2aNm2a0tPTC92+S5cucjgcmjRpkjIzM/X666/rpptuUlBQkMLCwtS/f3/98MMPOeufP39eL774olq2bKnAwEBVrVpVUVFR+v333wt9zfHx8XrmmWfUokULBQUFKTAwUC1atNCzzz6r48ePF7jd1Q7CdubMGT3zzDNq3LixAgICVLNmTQ0aNEixsbGF5ilMgwYNcj3n5MmTc3Jcvrwo78v69etztrvSldt/9dVX6tu3r6pXr66AgAA1b95ckydPzvPfw5VOnTqlKVOmqE2bNgoNDVVAQIAaNGigXr166Z133lFSUlKu5zt48KAk6YEHHsj1uq7MeLWf/cWLF/XGG2+offv2qlKligICAlS/fn0NHTpU33//fYF5s3++s2fPVlpamv7xj3/ohhtuUGBgoEJCQtStWzetWrWq0NcMALAhAwAAG3vwwQcNSTmXoKAgo2LFirmWHThwINc2EydONCQZnTt3zvcxn3rqqVzbV65c2fDx8TEkGbfccosxfvz4Arfv3LmzIckYP3680aNHD0OS4efnZwQGBubK+O233xonT540brrpJkOSERAQYFSoUCFnnbCwMOPgwYP55lu/fr1RuXLlnHUrVqyY6/GrVKlibNq0Kd9ts9dZt25dnvsOHDhg1K9fP2cdPz8/Izg4OGf86aefFrp9QVq3bm2Eh4cbvr6+hiQjMDDQCA8Pz7ls2bLFMIyrvy+GYRjr1q3LyXCly7efNm2a4XA4DIfDYVSuXNlwOBw523Xt2tXIyMjI9/FXr15tVKlSJWddHx+fXD9rScbSpUsNwzCMf/zjH0Z4eLjh5eVlSDKCg4Nzva7w8PBcj13Yz+7w4cNGy5Ytc9bx9fU1QkJCcm57eXkZb731Vr6Zs9+zf/3rX0abNm1ytg8KCsrZ3uFwGDExMQX+XAEA9sOeeQCAbW3evFnvv/++vLy8NHXqVJ06dUopKSk6d+6cTp48qdWrV2vYsGHy8/Mr8mPOmzdP//znPyVJ99xzjw4fPqwzZ84oJSVF//nPf/TNN99oxowZV32c6dOna+fOnVq4cKHOnj2rlJQUffPNN2rUqJHOnj2rJ554QsOHD9eZM2e0evVqnTt3TmfPntXatWtVvXp1JSQkaPz48Xke99ChQ+rfv78SExN13XXXafPmzTnbbty4Uc2aNdOZM2fUr18/HTlypMivOzMzU4MGDdLBgwdVpUoVLViwQOfOnVNSUpJ2796tNm3aaNiwYUV+vMt9++23io+PV/v27SVJTz/9tOLj43Mu2cvLyg8//KCxY8dq7NixSkhI0JkzZ5SYmKgXXnhBkrRu3Tp98MEHebbbuXOn+vXrpzNnzqhFixZauXKlzp8/rzNnzujcuXP69ttv9dRTT6lSpUq5XkfdunUlSW+++Wau1xUfH1+kvJmZmRowYIB27dqlkJAQzZkzR2fPnlViYqJ+//133XbbbcrKytLjjz+uL774osDHeeGFF3T48GEtW7ZM586dU0pKin7++We1bdtWhmHoiSeeyJlVAABwA1b/awIAACU1depUQ5IRGRlZrO0K2gOclZVlNG3a1JBk9OzZ08jKysqz7axZs3L2dha2Z15SvnvHv/rqq5z7K1SoYOzbty/POjExMTn3p6Wl5bpv5MiROXvfjx07lmfbQ4cO5exNf/TRR/PcrwL2Ds+fPz/nvrVr1+bZ7ty5c0bjxo1LtGc+W/bPZuLEifneX1Z75gt7jrvuusuQZPTo0SPPfR07djQkGU2bNjUSExOL8pIMw7i0Z3zWrFmFrlfQz27evHk5961atSrPdunp6Tl73Fu2bFng8/v7+xt79+7Nc39CQoIREBBgSDLmzJlT5NcFAHBt7JkHANhW5cqVJUknTpxQZmZmqR/v+++/1759+yRJ48ePz/d72cOGDSvSKe06duyojh075lneuXNn+fv7S5IGDhyoJk2a5FmnV69ekqQLFy7k5JEkwzC0YMECSdLIkSNVo0aNPNvWqVNHI0eOlGTOMiiq7HU7dOig7t2757m/YsWKevbZZ4v8eFby9/fX008/ne99/fr1kyT9+OOPuZbv27dPmzdvliS9/PLLCgkJcW7Iy8yfP1+S1K5du5z3/nI+Pj6aOHGiJGnXrl05ZwW40sCBA3XttdfmWV69enW1a9dOUt7XDQCwL8o8AMC2evTooYCAAO3cuVOdOnVSTEyMDhw4UOLH++677yRJvr6+BU79djgc6ty581Uf6+abb853ube3t6pVqyZJ+vOf/5zvOuHh4TnjM2fO5IwPHDig06dPSzJfe0F69uwpyTyQW1F/Hjt27JAkdevWrcB1CrvPlWQfFDA/tWrVkqScn2O2rVu3SjLfn969ezs34BWyf/aFvaddu3aVt7d3rvWv1KZNmwK3L+h1AwDsizIPALCtRo0a6b333lNQUJC2bdumhx9+WI0aNVJYWJiioqL06aefyjCMIj/eiRMnJElVq1Yt9Hv2tWvXvupjZX+vOj8+Pj6FrpN9vySlp6fnjBMSEoqU4fJzuF++TWGy1yvq47qyovzsMzIyci3P/n57tWrVFBgY6Lxw+SjKzz4gICDnH4EKek+L8rov/+8JAGBvlHkAgK3de++9OnjwoN555x1FRUWpbt26OnHihBYsWKD+/furc+fOSk5OLtJjZRf//KbX57eela6WsbjrFWX94j6WHVn5Gp31ngIA3BNlHgBge6GhoRoxYoTmzZunuLg4/fbbbxo7dqwcDoc2bdqkSZMmFelxwsLCJEknT55UWlpagesdPXq0LGIXW3Y+yTyqfUEOHz6cM65evXqxHvvybQt7XGfI3ntc2HngnXU09po1a0oyZ2ecO3fOKc9RkOyffWHv6cWLF3Xq1ClJRX9PAQDujTIPAHA7jRs3VnR0tO655x5J0po1a4q0XatWrSSZU5Gzv0N9JcMwtHHjxrIJWkwNGzZUaGioJOmrr74qcL21a9dKMr8u0LBhwyI9duvWrSWZp20ryH//+9+iRi2RKlWqSCq81G7fvt0pz519jITMzMxCT/+WHy8v88+pks7YyP7ZF/aerl+/PuerAQUdawEA4Fko8wAA20pNTS30/goVKkhSzoHDrubGG2/MObr8K6+8km85mzNnjg4ePFjMpGXD4XAoKipKkjRz5sx8z2N+9OhRzZw5U5J09913F/mxsx938+bNWr9+fZ77L1y4oH/84x8lSF10N9xwgyTzNXz99dd57k9ISNC7777rlOdu0qSJbrnlFknmmQyK+tUMSQoODpYkJSYmlui5hwwZIknatm2bvvzyyzz3Z2RkaMqUKZKkli1bqmXLliV6HgCAe6HMAwBsa9SoURo8eLAWL16c66BgZ8+e1TvvvKMPP/xQktSnT58iPZ7D4dDkyZMlSatXr9awYcNyptRfvHhRMTExGjFiRM4eZCuMHz9elStX1unTp9WjR49cMwi2bNmiHj16KDExUaGhoRo7dmyRH3fAgAE5MxMGDBigxYsX55zub+/everdu3eRD6ZXUu3bt1f9+vUlSffff7927NghwzCUlZWl9evXq0uXLsrKynLa87/55psKCAjQvn371KFDB61atSrngHHnz5/X9u3bNXLkyJyZD9myy/WiRYtynX2gqAYMGJBzJPrBgwfrk08+yXneAwcOaMCAAdq2bZskadq0aSV+fQAA90KZBwDYVnp6uhYuXKiBAwcqPDxclSpVUpUqVVSpUiX93//9n9LS0tSxY0c9//zzRX7Me+65R6NHj5YkffTRR6pTp45CQ0MVHByshx9+WO3atcs5j3tAQIAzXlah6tSpo2XLlikkJES7d+9Whw4dFBQUpKCgIHXs2FF79+5V5cqVtWzZsiIddT+bj4+PFi5cqLp16+r06dMaOHCgAgMDVblyZV133XXatm1bzj+OOIuXl5dmzpwpX19f/fLLL/rzn/+soKAgBQYGqmvXrsrIyNDbb7/ttOe/8cYb9emnnyokJES7du1S7969FRgYqNDQUAUGBqpt27aaOXOmzp49m2u7v/71r3I4HNq6dauqV6+uWrVqqUGDBmrQoEGRntfb21uLFy9WixYtlJSUpHvvvVdBQUGqUqWKGjVqpOXLl8vLy0tvvvlmuZ82DwDguijzAADbmjBhgt566y3deeeduvbaa+Xj46OzZ88qLCxMPXv21Pvvv6/169cX+1Rjr7/+upYsWaIuXbqoUqVKSk1NVfPmzfWPf/xDq1evzjlAWuXKlZ3wqq6uc+fO+vnnn/XUU0+pefPmysrKkmEYat68uZ5++mnt3btXnTp1KvbjNmrUSN9//73GjBmjhg0byjAMBQQEaODAgdq6davuuOMOJ7ya3Hr16qVNmzbptttuU5UqVZSZmam6detq7Nixio2NVY0aNZz6/JGRkdq3b5+ef/553XTTTapQoYIuXLigBg0aqFevXpo5c6a6deuWa5tbbrlFn3/+uXr06KGQkBAdP35cBw8eLNbXMWrXrq0dO3botddeU9u2bVWhQgWdP39edevW1X333afY2Fg9/vjjZf1yAQA25jBc4fw6AADYSIcOHbR161ZNmTJFEyZMsDoOAADwQOyZBwCgGDZs2JDzPfVbb73V4jQAAMBTUeYBALjCo48+qtmzZys+Pj7niPaJiYmaOXOm+vXrJ0nq1q0bpwgDAACWYZo9AABXuPHGG/XDDz9Ikvz9/VWxYkUlJibmFPvrrrtOX375ZbEOMAcAAFCWKPMAAFxh+fLlWrp0qb755hsdP35cSUlJCg4OVosWLXTXXXfpr3/9qypWrGh1TAAA4MEo8wAAAAAA2AzfmQcAAAAAwGZ8rA7gqrKysnT06FFVqlRJDofD6jgAAAAAADdnGIZSUlJUq1YteXkVvu+dMl+Ao0ePqm7dulbHAAAAAAB4mEOHDqlOnTqFrkOZL0ClSpUkmT/E4OBgi9MAAAAAANxdcnKy6tatm9NHC0OZL0D21Prg4GDKPAAAAACg3BTlq94cAA8AAAAAAJuhzAMAAAAAYDOUeQAAAAAAbIYyDwAAAACAzVDmAQAAAACwGco8AAAAAAA2Q5kHAAAAAMBmKPMAAAAAANgMZR4AAAAAAJuhzAMAAAAAYDOUeQAAAAAAbIYyDwAAAACAzVDmAQAAAACwGco8AAAAAAA2Q5kHAAAAAMBmKPMAAAAAANgMZR4AAAAAAJuhzAMAAAAAYDOUeQAAAAAAbIYyDwAAAACAzVDmAQAAAACwGVuU+Y0bN+r2229XrVq15HA4tGzZsqtus2HDBkVERCggIECNGjXSO++84/ygAAAAAACUA1uU+XPnzumGG27Qv//97yKtf+DAAfXp00edOnXSzp07NX78eD3++ONavHixk5MCAAAAAOB8PlYHKIrevXurd+/eRV7/nXfeUb169fTGG29Ikpo3b64dO3bo1Vdf1YABA5yUEgAAAAAsYhiXLllZeS8FLS/svqstv/w5r8xw+W1njEu6fY8eUnBw2f3cLWSLMl9c27ZtU2RkZK5lvXr1UkxMjNLT0+Xr65tnm9TUVKWmpubcTk5OdnpOAAAAAE6QlSWlp1t7SUsr3vqZmaUr4NllFYX76SepZUurU5QJtyzz8fHxCg8Pz7UsPDxcGRkZOnnypGrWrJlnm+joaE2ePLm8IgIAAAC4nGFI585JKSlScvKl68vHRV129qzVr8YevLwkh8O8vvJyteXZ15I5djhyj6+87YxxSbapWLHsfn4Wc8syL0mO7Dftf4z//UvVlcuzjRs3TmPGjMm5nZycrLp16zovIAAAAGB3hiFdvFj68p2SYl6yspyX1dtb8vXNffHzy7vMmZfCns/b27xcrUyXpoBfuQy25pZlvkaNGoqPj8+1LCEhQT4+PqpatWq+2/j7+8vf37884gEAAACuKyNDOnhQ+u03ad8+8/ro0YILeWZm2T6/w2F+pzk4WKpUKe+4KMsqVZL8/XOXZcor3Ixblvl27dppxYoVuZZ9+eWXat26db7flwcAAAA8Snq6Wdizy/rlxf3AAbPQF1dQUNGK99Xur1iR4g0UgS3K/NmzZ/Xbb7/l3D5w4IC+//57hYaGql69eho3bpyOHDmiDz/8UJI0cuRI/fvf/9aYMWM0fPhwbdu2TTExMZo7d65VLwEAAAAoX+npZjG/sqxnF/bC9qj7+0tNmly61Kt3qXTnV8KDgi59fxpAubBFmd+xY4e6du2aczv7u+3Dhg3T7NmzdezYMcXFxeXc37BhQ61cuVJPPvmk3n77bdWqVUtvvfUWp6UDAACAe0lLu1TYr9zLfvBg4YU9IOBSWW/aNPd17dqUc8DFOQyDcxjkJzk5WSEhIUpKSlKwm5yHEAAAADaUmpq3sGdfHzxY+EHjKlTIv6w3aSLVqkVhB1xMcXqoLfbMAwAAAG7t4kWzsOf3Hfa4uMILe2Bg3j3s2eOaNfn+OeCmKPMAAABAebl4Ufrvf6W9e3MX97g48zRvBQkKyr+sN2ki1ahBYQc8EGUeAAAAcCbDkL7+WvrgA2n+fCkxMf/1KlUqeEp8eDiFHUAulHkAAADAGQ4elD76SPrwQ3MvfLY6daQOHfLuZa9encIOoMgo8wAAAEBZOXtWWrzY3Au/bt2l5RUrSgMHSsOGSV26cOA5AKVGmQcAAABKIytLWr/eLPCLF0vnzl26r2tXs8APGGB+7x0AyghlHgAAACiJX381C/xHH0mHDl1a3rSpNHSodN99Uv361uUD4NYo8wAAAEBRnTljHsTugw/Mg9plCwmRhgwx98K3bct33wE4HWUeAAAAKEx6urR6tVngly+X0tLM5d7eUq9eZoG/4w4pIMDanAA8CmUeAAAAyM8PP5gF/uOPpYSES8v/9CezwN97r3mOdwCwAGUeAAAAyHb8uPTJJ2aJ/+GHS8urVzfL+7Bh0g03MI0egOUo8wAAAPBsFy9KK1aY54P/4gspM9Nc7ucn3X67WeBvvVXy9bU2JwBchjIPAAAAz2MY0vbt5h74efOkxMRL9918s1nghwyRQkMtiwgAhaHMAwAAwHPExUlz5pgl/tdfLy2vU8c8ldzQodK111qXDwCKiDIPAAAA93b2rLRkiVng160z98pLUsWK0l13mXvhu3Y1j04PADZBmQcAAID7ycqSNmwwC/yiRdK5c5fu69zZLPADB0qVKlmXEQBKgTIPAAAA97Fvn1ngP/rInFKfrXFjcwr9ffdJDRtalw8AyghlHgAAAPaWmCjNn2+W+G3bLi0PDpaiosy98O3bczo5AG6FMg8AAAB7ysiQnnlGmjFDSk01l3l5SZGRZoHv10+qUMHajADgJJR5AAAA2M/58+ap41asMG+3aGEW+HvvlWrVsjYbAJQDyjwAAADs5fRp6fbbpa1bJX9/81RzAwYwjR6AR6HMAwAAwD4OHZJuvVXas0eqXFlavlzq1MnqVABQ7ijzAAAAsIfdu80if/iwOZV+9WqpZUurUwGAJbysDgAAAABc1dat5h74w4ela681b1PkAXgwyjwAAABc2/LlUvfu0pkzUtu20ubNUv36VqcCAEtR5gEAAOC6YmKkO++ULl6U+vaV1q6Vqla1OhUAWI4yDwAAANdjGNJLL0kPPyxlZUn33y8tXSoFBlqdDABcAmUeAAAAriUzU3rsMelvfzNvjxsnvf++5OtrbS4AcCEczR4AAACuIzVV+stfpEWLzPPGv/GG9PjjVqcCAJdDmQcAAIBrSEoyvx+/bp25F/6jj6SoKKtTAYBLoswDAADAeseOSb17Sz/8IAUFmd+P79HD6lQA4LIo8wAAALDWvn1SZKT0xx9SWJj0xRdSq1ZWpwIAl8YB8AAAAGCdHTukDh3MIt+4sbR1K0UeAIqAMg8AAABrfPml1KWLdOKEWeC3bDELPQDgqijzAAAAKH8ffyz17SudO2d+N379eik83OpUAGAblHkAAACUr9deM08/l5EhDRkiff65VKmS1akAwFYo8wAAACgfWVnSM89ITz1l3h492txD7+dnaSwAsCOOZg8AAADnS0+XHnxQmjPHvD11qlnsHQ5rcwGATVHmAQAA4Fxnz0qDBkmrVkne3lJMjDRsmNWpAMDWKPMAAABwnhMnzAPdffutVLGitHCh1KeP1akAwPYo8wAAAHCOP/6QIiOlffuk0FDzQHdt21qdCgDcAmUeAAAAZe+HH6Rbb5Xi46V69aTVq6Vrr7U6FQC4DY5mDwAAgLK1fr10yy1mkW/ZUtq6lSIPAGWMMg8AAICys3ix1KuXlJwsdeokbdok1a5tdSoAcDuUeQAAAJSNGTPMo9anpUl33mlOra9c2epUAOCWKPMAAAAoHcOQXnhBeuQRczxihHnU+goVrE4GAG6LA+ABAACg5DIyzBL/7rvm7UmTzGLvcFgaCwDcHWUeAAAAJXPhgnT33dKnn0peXtL06eZeeQCA01HmAQAAUHxnzkh33CFt3iz5+0tz55rfkwcAlAvKPAAAAIrn8GHzHPK7d0shIdLy5eap6AAA5YYyDwAAgKLbu9c89dyhQ1KtWtKqVdKf/mR1KgDwOBzNHgAAAEWzbZvUsaNZ5Js1k7ZupcgDgEUo8wAAALi6zz6TuneXTp+W2rQxvytfv77VqQDAY1HmAQAAULhZs6T+/c2j1/fpI331lVStmtWpAMCjUeYBAACQP8OQoqOlBx+UMjOlYcOkZcukwECrkwGAx6PMAwAAIK+sLGn0aGn8ePP22LHmHnpfX0tjAQBMHM0eAAAAuaWmSkOHSgsWmLffeEN64glLIwEAcqPMAwAA4JLkZOnOO6X//tfcC//hh9KQIVanAgBcgTIPAAAA0+nTUo8e0s6dUlCQtHSpeRsA4HIo8wAAADC98IJZ5MPCpJUrpYgIqxMBAArAAfAAAAAg7d8vzZxpjufNo8gDgIujzAMAAMDcK5+RIfXqJXXtanUaAMBVUOYBAAA83Y8/Sp98Yo5fftnaLACAIqHMAwAAeLrnn5cMQ4qKklq1sjoNAKAIKPMAAACebPNm6bPPJG9v6e9/tzoNAKCIKPMAAACeyjCksWPN8UMPSU2bWpsHAFBklHkAAABPtXKltGWLFBBgHgAPAGAblHkAAABPlJUljR9vjh9/XKpd29o8AIBiocwDAAB4onnzzKPYh4RIzz1ndRoAQDFR5gEAADxNWpo0YYI5fu45KTTU2jwAgGKjzAMAAHia996T9u+XwsPNKfYAANuhzAMAAHiSc+ekKVPM8QsvSIGB1uYBAJQIZR4AAMCTvPmmdPy41KiR9PDDVqcBAJQQZR4AAMBTnD4tTZtmjv/+d8nPz9o8AIASo8wDAAB4iqlTpaQk6frrpSFDrE4DACgFyjwAAIAnOHJEeustc/zyy5IXfwYCgJ3xf3EAAABPMGWKdPGi1LGj1KeP1WkAAKVEmQcAAHB3v/4qxcSY4+hoyeGwNg8AoNQo8wAAAO5uwgQpM1O67TZzzzwAwPYo8wAAAO7su++kBQvMvfEvvWR1GgBAGaHMAwAAuLPx483re+81j2IPAHALlHkAAAB3tW6dtHq15OMjTZ5sdRoAQBmizAMAALgjw5DGjTPHI0ZIjRpZmwcAUKYo8wAAAO7o00+l7dulihWlv/3N6jQAgDJGmQcAAHA3mZnS88+b4yeflGrUsDYPAKDMUeYBAADczZw50p49Umio9MwzVqcBADgBZR4AAMCdpKZKL7xgjseOlUJCrM0DAHAKyjwAAIA7eecdKS5OqlVLGjXK6jQAACehzAMAALiLlBTpxRfN8aRJUoUKlsYBADgPZR4AAMBdvPaadPKkdM010gMPWJ0GAOBElHkAAAB3cOKE9Oqr5vjFFyUfH2vzAACcijIPAADgDqKjpbNnpYgIacAAq9MAAJyMMg8AAGB3cXHS22+b45dflrz4Ew8A3B3/pwcAALC7SZOktDSpa1epZ0+r0wAAygFlHgAAwM727JE++MAcR0dLDoe1eQAA5YIyDwAAYGd/+5uUlSXdeafUpo3VaQAA5YQyDwAAYFfffCMtXWp+Rz77/PIAAI9AmQcAALAjw5DGjjXHQ4dK111nbR4AQLmizAMAANjR2rXSunWSn595ADwAgEehzAMAANhNVpY0bpw5fuQRqX59a/MAAModZR4AAMBuFi+WYmOloCBp/Hir0wAALGCbMj99+nQ1bNhQAQEBioiI0KZNmwpd/+OPP9YNN9ygihUrqmbNmnrggQd06tSpckoLAADgJBkZ5hHsJenpp6Xq1a3NAwCwhC3K/Pz58zV69Gg9//zz2rlzpzp16qTevXsrLi4u3/U3b96soUOH6qGHHtLu3bu1cOFCffvtt3r44YfLOTkAAEAZmz1b+vVXqVo1acwYq9MAACxiizL/2muv6aGHHtLDDz+s5s2b64033lDdunU1Y8aMfNf/+uuv1aBBAz3++ONq2LChOnbsqBEjRmjHjh3lnBwAAKAMXbhw6WB3zz8vVapkaRwAgHVcvsynpaUpNjZWkZGRuZZHRkZq69at+W7Tvn17HT58WCtXrpRhGDp+/LgWLVqkvn37Fvg8qampSk5OznUBAABwKW+/LR05ItWrJ40caXUaAICFXL7Mnzx5UpmZmQoPD8+1PDw8XPHx8flu0759e3388ceKioqSn5+fatSoocqVK+tf//pXgc8THR2tkJCQnEvdunXL9HUAAACUSmKi9PLL5njyZCkgwNI4AABruXyZz+ZwOHLdNgwjz7Jse/bs0eOPP64XXnhBsbGxWrVqlQ4cOKCRhfwL9rhx45SUlJRzOXToUJnmBwAAKJVXX5XOnJGuu0667z6r0wAALOZjdYCrqVatmry9vfPshU9ISMiztz5bdHS0OnTooGeeeUaSdP311yswMFCdOnXSiy++qJo1a+bZxt/fX/7+/mX/AgAAAEorPl56/XVz/NJLkre3tXkAAJZz+T3zfn5+ioiI0Jo1a3ItX7Nmjdq3b5/vNufPn5eXV+6X5v2/X3qGYTgnKAAAgLO89JJ0/rzUpo3Ur5/VaQAALsDly7wkjRkzRu+9957ef/997d27V08++aTi4uJyps2PGzdOQ4cOzVn/9ttv15IlSzRjxgzt379fW7Zs0eOPP66bb75ZtWrVsuplAAAAFN/+/dLMmeY4Oloq4GuGAADP4vLT7CUpKipKp06d0pQpU3Ts2DG1bNlSK1euVP369SVJx44dy3XO+fvvv18pKSn697//raeeekqVK1dWt27dNHXqVKteAgAAQMlMnCilp0uRkVLXrlanAQC4CIfBvPN8JScnKyQkRElJSQoODrY6DgAA8EQ//ijdeKNkGNKOHVJEhNWJAABOVJweaotp9gAAAB7p+efNIj94MEUeAJALZR4AAMAVbdkiffaZeeT6v//d6jQAABdDmQcAAHA1hiGNHWuOH3xQuuYaa/MAAFwOZR4AAMDVfPGFtHmzFBAgvfCC1WkAAC6IMg8AAOBKsrKkcePM8WOPSXXqWJsHAOCSKPMAAACuZN488yj2ISGXptoDAHAFyjwAAICrSEuTJkwwx888I4WGWpsHAOCyKPMAAACuIiZG2r9fCg+XnnjC6jQAABdGmQcAAHAF585JU6aY4wkTpKAga/MAAFwaZR4AAMAVvPWWFB8vNWwoDR9udRoAgIujzAMAAFjt9Glp6lRz/Pe/S35+1uYBALg8yjwAAIDVpk6VkpKk66+X7r7b6jQAABugzAMAAFjpyBFzir0kvfSS5MWfZwCAq+O3BQAAgJX+/nfp4kWpQwepb1+r0wAAbIIyDwAAYJV9+6T33jPHr7wiORzW5gEA2AZlHgAAwCoTJkiZmeYe+Y4drU4DALARyjwAAIAVvvtOmj/f3Bv/8stWpwEA2AxlHgAAwArjx5vXd99tHsUeAIBioMwDAACUt/XrpdWrJR8facoUq9MAAGyIMg8AAFCeDEMaN84c//WvUuPG1uYBANgSZR4AAKA8LV8uff21VLGi9Le/WZ0GAGBTlHkAAIDykpl56bvyo0dLNWtaGgcAYF+UeQAAgPIyZ460Z49UpYr0zDNWpwEA2BhlHgAAoDykpkovvGCOx46VKle2NA4AwN4o8wAAAOVh5kwpLk6qVUsaNcrqNAAAm6PMAwAAOFtKivTii+Z44kTz4HcAAJQCZR4AAMDZXn9dOnFCatpUeuABq9MAANwAZR4AAMCZTpyQXn3VHL/4ouTra20eAIBboMwDAAA4U3S0Oc3+ppukgQOtTgMAcBOUeQAAAGeJi5OmTzfH0dGSF396AQDKBr9RAAAAnGXyZPOUdF26SJGRVqcBALgRyjwAAIAzJCVJc+aY45dekhwOa/MAANwKZR4AAMAZli2T0tKk666T2rWzOg0AwM1Q5gEAAJxh3jzzesgQ9soDAMocZR4AAKCsnTwprVljjqOirM0CAHBLlHkAAICytnixlJkptWolXXON1WkAAG6IMg8AAFDWLp9iDwCAE1DmAQAAytLRo9KGDeZ48GBrswAA3BZlHgAAoCwtXCgZhtS+vVS/vtVpAABuijIPAABQlphiDwAoB5R5AACAsnLggPT115KXlzRokNVpAABujDIPAABQVubPN6+7dJFq1LA0CgDAvVHmAQAAygpT7AEA5YQyDwAAUBb27pV++EHy8ZHuusvqNAAAN0eZBwAAKAvZU+wjI6WqVa3NAgBwe5R5AACA0jIMptgDAMoVZR4AAKC0fvhB+uUXyd9f6tfP6jQAAA9AmQcAACit7L3yfftKwcHWZgEAeATKPAAAQGkwxR4AYAHKPAAAQGls3y4dPCgFBpp75gEAKAeUeQAAgNLI3ivfr59UsaK1WQAAHoMyDwAAUFKZmdKCBeaYKfYAgHJEmQcAACipTZukY8ekypXN88sDAFBOKPMAAAAllT3F/q67zNPSAQBQTijzAAAAJZGeLi1aZI6ZYg8AKGeUeQAAgJL46ivp1CmpenWpa1er0wAAPAxlHgAAoCSyp9gPGiT5+FibBQDgcSjzAAAAxXXxorR0qTlmij0AwAKUeQAAgOJatUpKTpZq15Y6dLA6DQDAA1HmAQAAiit7in1UlOTFn1MAgPLHbx8AAIDiOHdOWrHCHDPFHgBgEco8AABAcaxYIZ0/LzVqJLVubXUaAICHoswDAAAUR/YU+yFDJIfD2iwAAI9FmQcAACiqxETpiy/MMVPsAQAWoswDAAAU1bJlUlqadN11UsuWVqcBAHgwyjwAAEBRZU+xv/tuptgDACxFmQcAACiKEyektWvNcVSUtVkAAB6PMg8AAFAUixdLmZlSRITUtKnVaQAAHo4yDwAAUBSXH8UeAACLUeYBAACu5sgRaeNGczx4sLVZAAAQZR4AAODqFi6UDEPq0EGqV8/qNAAAUOYBAACuiin2AAAXQ5kHAAAozIED0vbtkpeXNHCg1WkAAJBEmQcAACjc/PnmddeuUo0a1mYBAOB/KPMAAACFYYo9AMAFUeYBAAAKsnev9MMPko+PdNddVqcBACAHZR4AAKAg2VPse/WSQkOtzQIAwGUo8wAAAPkxDKbYAwBcFmUeAAAgPz/8IP3yixQQIN1xh9VpAADIhTIPAACQn+y98n37SsHB1mYBAOAKlHkAAIArMcUeAODiKPMAAABX2r5dOnhQCgqS+vSxOg0AAHlQ5gEAAK6UvVe+Xz+pYkVrswAAkA/KPAAAwOUyM6UFC8wxU+wBAC6KMg8AAHC5TZukY8ekypWlyEir0wAAkC/KPAAAwOWyp9gPGCD5+VmbBQCAAlDmAQAAsqWnS4sWmWOm2AMAXBhlHgAAINtXX0mnTklhYVKXLlanAQCgQJR5AACAbNlT7AcNknx8rM0CAEAhKPMAAACSdPGitHSpOWaKPQDAxVHmAQAAJGnVKik5WapTR2rf3uo0AAAUijIPAAAgXZpiHxUlefEnEgDAtfGbCgAA4Nw5acUKc8wUewCADVDmAQAAVqyQzp+XGjeWIiKsTgMAwFVR5gEAALKn2A8ZIjkc1mYBAKAIKPMAAMCzJSZKX3xhjpliDwCwCco8AADwbMuWSWlpUosWUsuWVqcBAKBIKPMAAMCzXT7FHgAAm6DMAwAAz3XihLR2rTmOirI2CwAAxUCZBwAAnmvxYikz0zyCfdOmVqcBAKDIKPMAAMBzzZ1rXjPFHgBgM7Yp89OnT1fDhg0VEBCgiIgIbdq0qdD1U1NT9fzzz6t+/fry9/dX48aN9f7775dTWgAA4PIOH5ay/54YPNjaLAAAFJOP1QGKYv78+Ro9erSmT5+uDh06aObMmerdu7f27NmjevXq5bvN4MGDdfz4ccXExKhJkyZKSEhQRkZGOScHAAAua+FCyTCkDh2kAv6eAADAVTkMwzCsDnE1bdq0UatWrTRjxoycZc2bN1f//v0VHR2dZ/1Vq1ZpyJAh2r9/v0JDQ0v0nMnJyQoJCVFSUpKCg4NLnB0AALioNm2kb76R/v1v6dFHrU4DAECxeqjLT7NPS0tTbGysIiMjcy2PjIzU1q1b891m+fLlat26taZNm6batWvrmmuu0dNPP60LFy4U+DypqalKTk7OdQEAAG5q/36zyHt5SQMHWp0GAIBic/lp9idPnlRmZqbCw8NzLQ8PD1d8fHy+2+zfv1+bN29WQECAli5dqpMnT+qRRx7R6dOnC/zefHR0tCZPnlzm+QEAgAuaP9+87tZNuuJvDAAA7MDl98xnczgcuW4bhpFnWbasrCw5HA59/PHHuvnmm9WnTx+99tprmj17doF758eNG6ekpKScy6FDh8r8NQAAABcxb555zVHsAQA25fJ75qtVqyZvb+88e+ETEhLy7K3PVrNmTdWuXVshISE5y5o3by7DMHT48GE1zec8sv7+/vL39y/b8AAAwPXs2SP9+KPk6yvdeafVaQAAKBGX3zPv5+eniIgIrVmzJtfyNWvWqH379vlu06FDBx09elRnz57NWfbrr7/Ky8tLderUcWpeAADg4rKn2PfqJZXwQLkAAFjN5cu8JI0ZM0bvvfee3n//fe3du1dPPvmk4uLiNHLkSEnmFPmhQ4fmrH/PPfeoatWqeuCBB7Rnzx5t3LhRzzzzjB588EFVqFDBqpcBAACsZhhMsQcAuAWXn2YvSVFRUTp16pSmTJmiY8eOqWXLllq5cqXq168vSTp27Jji4uJy1g8KCtKaNWv02GOPqXXr1qpataoGDx6sF1980aqXAAAAXMH330u//ioFBEh33GF1GgAASswW55m3AueZBwDADT33nDRtmnk6uoULrU4DAEAubnWeeQAAgDLBFHsAgBuhzAMAAM/w9ddSXJwUFCT16WN1GgAASoUyDwAAPEP2Xvn+/SUOiAsAsDnKPAAAcH+ZmdKCBeaYKfYAADdAmQcAAO5v40YpPl6qUkXq2dPqNAAAlBplHgAAuL/sKfYDBkh+ftZmAQCgDFDmAQCAe0tPlxYtMsdMsQcAuAnKPAAAcG9r10qnT0vh4VKXLlanAQCgTFDmAQCAe8ueYj9okOTtbW0WAADKCGUeAAC4r4sXpaVLzTFT7AEAboQyDwAA3NcXX0gpKVLdulK7dlanAQCgzFDmAQCA+8qeYh8VJXnxZw8AwH3wWw0AALins2elFSvMMVPsAQBuhjIPAADc04oV0oULUpMmUqtWVqcBAKBMUeYBAIB7yp5iP2SI5HBYmwUAgDJGmQcAAO7nzBnz4HcSU+wBAG6JMg8AANzPsmVSerrUsqXUooXVaQAAKHOUeQAA4H4un2IPAIAboswDAAD3kpAgffWVOY6KsjYLAABOQpkHAADuZfFiKTNTat3aPJI9AABuiDIPAADcC1PsAQAegDIPAADcx+HD0qZN5njwYGuzAADgRJR5AADgPhYulAxD6thRqlvX6jQAADgNZR4AALgPptgDADwEZR4AALiH/fulb76RvLykgQOtTgMAgFNR5gEAgHuYP9+87tZNCg+3NgsAAE5GmQcAAO6BKfYAAA9CmQcAAPa3Z4/044+Sr690111WpwEAwOko8wAAwP6yp9jfeqtUpYq1WQAAKAeUeQAAYG+GwRR7AIDH8SmLBzl//ry+/fZbHT16VCdOnNDFixdVtWpVVa9eXc2bN1fTpk3L4mkAAADy+v576ddfpQoVpDvusDoNAADlosRl/vDhw4qJidGqVav03XffKSMjo8B1w8PD1blzZ917773q27evHA5HSZ8WAAAgt+y98rfdJgUFWZsFAIBy4jAMwyjOBps3b9a0adP0xRdfKCsrS5dv7uXlpZCQEFWoUEGnT5/WxYsXcz+Zw6HatWtrxIgReuKJJxTkwr9wk5OTFRISoqSkJAUHB1sdBwAA5McwpAYNpLg4afFiDn4HALC14vTQIpf5ffv26dlnn9Xy5ctlGIa8vb3VrVs33XLLLWrTpo1uuukmhYaG5trrnpqaqv3792v79u3avn27Pv/8cx0+fFgOh0PVq1fX5MmTNXz4cHl5ud5X9ynzAADYwLZtUvv2UqVK0vHj5lR7AABsqjg9tMjT7Fu0aKGMjAxdd911Gj58uO6++26FhYUVuo2/v7+aN2+u5s2b6/7775dhGNqwYYM+/PBDffLJJ3rkkUd0+vRpjRs3rqgxAAAALsmeYt+/P0UeAOBRirxL/JprrtHcuXP1008/6Yknnrhqkc+Pw+FQly5d9P777+u3337TiBEjXHKvPAAAsIHMTGnBAnPMUewBAB6myNPsDcNwyoHrnPW4pcU0ewAAXNy6dVK3blJoqHTsmOTnZ3UiAABKpTg9tMi7xZ1VuF2xyAMAABvInmI/YABFHgDgcZjjDgAA7Cc9XVq0yBwzxR4A4IFKVea7deumbt266bXXXivS+gMGDFD37t1L85QAAADS2rXS6dNSeLjUubPVaQAAKHdFPpp9ftavXy+Hw6ENGzZoz549euedd+TjU/BDbt26VQkJCaV5SgAAgEtT7AcPlry9rc0CAIAFSj3N3sfHR15eXpo1a5Z69eqlxMTEMogFAABQgIsXpaVLzTFT7AEAHqrUZb5q1apasWKFAgMDtX79erVt21a///57WWQDAADI64svpJQUqV49qW1bq9MAAGCJMjkA3q233qrNmzerdu3a+vXXX9WmTRtt3LixLB4aAAAgt+wp9lFRkhfH8gUAeKYy+w14/fXXa/v27WrVqpVOnz6tyMhIzZ49u6weHgAAQDp7VlqxwhwzxR4A4MHK9J+za9asqU2bNqlfv35KS0vTQw89pHHjxpXlUwAAAE+2YoV04YLUtKl0001WpwEAwDJlPjetQoUKWrJkiZ588kkZhqFp06Zp4MCBunDhQlk/FQAA8DRz55rXQ4ZIDoe1WQAAsJBTvmjmcDj0z3/+UzNmzJC3t7eWLl2qW265hUIPAABK7swZadUqc8wUewCAh3PqUWNGjBihzz77TJUqVdJ3332n5ORkZz4dAABwZ0uXSunp0p/+JF13ndVpAACwVKnKfL169VS3bt1C14mMjNSWLVuuuh4AAEChLj+KPQAAHs6nNBv/8ccfRVqvRYsWio2N1a5du0rzdAAAwFMlJEhffWWOmWIPAEDpynxxVK1aVZ07dy6vpwMAAO5k0SIpK0v685+lxo2tTgMAgOWc+p15AACAMpE9xZ698gAASCpGmV+0aFGZP/mRI0e0bdu2Mn9cAADgRg4fljZtMseDB1ubBQAAF1HkMj948GDdeOONWrp0qQzDKNWTHjp0SKNGjVKTJk20Zs2aUj0WAABwcwsWmNedOkl16libBQAAF1HkMn/PPffop59+0sCBA1W7dm09/fTTio2NLXKxP3HihGJiYtStWzc1bNhQ06dPV3h4uLp06VLS7AAAwBMwxR4AgDwcRjF2s+/YsUNPPfWUNm3aJIfDIUkKDAxUq1atdMMNN6h69eoKDQ2Vv7+/zpw5o9OnT2v//v365ptvdPDgQUmSYRgKDg7W2LFj9eSTT8rf3985r6yUkpOTFRISoqSkJAUHB1sdBwAAz/T771KTJpKXl3TsmBQWZnUiAACcpjg9tFhHs2/durU2bNig7du3a8aMGVq4cKHOnj2rjRs3alP2d9mucPm/Fdxwww0aMWKE/vKXvygoKKg4Tw0AADzR/PnmdffuFHkAAC5TrD3zV0pJSdGGDRu0ceNGbd++XUePHtWJEyd08eJFVa1aVdWrV9d1112nW265RV26dNG1115bltmdij3zAAC4gOuvl376SYqJkR580Oo0AAA4VXF6aKnKvDujzAMAYLHdu6WWLSVfX+n4calKFasTAQDgVMXpoZxnHgAAuKbsKfa33kqRBwDgCpR5AADgegyDo9gDAFCIYh0A70oPFvO7awEBAapcubJatGih7t27q0aNGqV5egAA4K527pT27ZMqVJDuuMPqNAAAuJxSlfnZs2fnnKKuKAzDyFnfx8dHw4YN02uvvcaR7QEAQG7Ze+Vvu03i7wQAAPIoVZkfOnSoHA6Hli9frjNnzqhixYqKiIhQ7dq1ZRiGjh49qtjYWJ0/f16hoaG67bbblJiYqO+++06HDx9WTEyM9u3bp7Vr18rb27usXhMAALCzrKxL35dnij0AAPkq1XfmZ8+erbS0NCUmJmrSpEmKj4/Xhg0b9Mknn2ju3LnasGGDjh8/rsmTJysxMVGStGzZMsXFxWn27Nny9fXVxo0bNWfOnLJ4LQAAwB18/bUUFydVqiT17m11GgAAXFKpyvy7776refPm6eWXX9YLL7yQ73T5wMBATZgwQS+99JI++ugjzZ49W5K5V3/SpEkyDENz584tTQwAAOBOsqfY9+9vfmceAADkUarzzLdt21axsbE6c+bMVb/3fvbsWVWpUkU333yztmzZIklKSEhQjRo1FBYWpvj4+JLGcArOMw8AgAUyM6Xatc3zyn/+udSnj9WJAAAoN+V2nvmff/5ZISEhRTqAXVBQkIKDg7V79+6cZWFhYQoJCcmZgg8AADzchg1mkQ8NlXr0sDoNAAAuq1RlPisrS4mJiTpz5sxV1z1z5oySkpKUlZWVa3l6ejpHswcAAKbsKfYDBkh+ftZmAQDAhZWqzLds2VKGYSg6Ovqq677yyivKyspSixYtcpadOnVK58+fV1hYWGliAAAAd5CWJi1ebI45ij0AAIUqVZkfPny4DMPQP//5T40YMUIHDx7Ms05cXJxGjhypV199VQ6HQ3/9619z7lu/fr0k6aabbipNDAAA4A7WrpVOn5Zq1JA6d7Y6DQAALq1U55l/4IEHtHr1ai1YsEDvvfee3nvvPdWrV0+1atWSw+HQ0aNHcwq+YRgaNGiQHnjggZztFy9erJCQEPXmtDMAACB7iv3gwZK3t7VZAABwcaU6mr1kfm9+2rRpeuWVV5ScnJzvOsHBwXruuef07LPPytsmv5w5mj0AAOXowgUpPFxKSZG2bpXatbM6EQAA5a44PbTUZT7bhQsXtHr1an333Xc6ceKEDMNQWFiYWrVqpcjISFWsWLEsnqbcUOYBAChHS5aYB72rX186cEByOKxOBABAuStODy3VNPvLVahQQf3791f//v3L6iEBAICnyJ5iHxVFkQcAoAhKdQA8AACAUktJkT77zBxzFHsAAIqEMg8AAKy1YoX5nflrrpFuvNHqNAAA2AJlHgAAWCt7iv2QIUyxBwCgiCjzAADAOmfOSKtWmeOoKGuzAABgI5R5AABgnaVLpfR06frrpeuuszoNAAC2QZkHAADWuXyKPQAAKDLKPAAAsEZCgvTVV+aYKfYAABQLZR4AAFhj0SIpK0u6+WapUSOr0wAAYCuUeQAAYI25c81rptgDAFBslHkAAFD+Dh2SNm82T0U3eLDVaQAAsB3KPAAAKH8LFpjXnTpJtWtbmwUAABuizAMAgPLHUewBACgVyjwAAChfv/0m7dgheXtLAwZYnQYAAFuizAMAgPI1f7553b27FBZmbRYAAGyKMg8AAMoXU+wBACg1yjwAACg/u3aZF19f6c47rU4DAIBtUeYBAED5yZ5i37u3VLmypVEAALAzyjwAACgfhsEUewAAyghlHgAAlI/vvjOPZF+hgnT77VanAQDA1ijzAACgfGTvlb/9dikoyNosAADYHGUeAAA4X1bWpe/LM8UeAIBSo8wDAADn27ZNOnRIqlTJPPgdAAAoFco8AABwvuwp9nfeKQUEWJsFAAA3QJkHAADOlZEhLVhgjpliDwBAmaDMAwAA59qwQUpIkEJDpR49rE4DAIBbsE2Znz59uho2bKiAgABFRERo06ZNRdpuy5Yt8vHx0Y033ujcgAAAIH/ZU+wHDpR8fa3NAgCAm7BFmZ8/f75Gjx6t559/Xjt37lSnTp3Uu3dvxcXFFbpdUlKShg4dqu7du5dTUgAAkEtamrR4sTlmij0AAGXGYRiGYXWIq2nTpo1atWqlGTNm5Cxr3ry5+vfvr+jo6AK3GzJkiJo2bSpvb28tW7ZM33//fZGfMzk5WSEhIUpKSlJwcHBp4gMA4Lk+/1y67TapRg3p8GHJ29vqRAAAuKzi9FCX3zOflpam2NhYRUZG5loeGRmprVu3FrjdrFmz9Pvvv2vixIlFep7U1FQlJyfnugAAgFLKnmI/eDBFHgCAMuTyZf7kyZPKzMxUeHh4ruXh4eGKj4/Pd5t9+/Zp7Nix+vjjj+Xj41Ok54mOjlZISEjOpW7duqXODgCAR7twQVq2zBwzxR4AgDLl8mU+m8PhyHXbMIw8yyQpMzNT99xzjyZPnqxrrrmmyI8/btw4JSUl5VwOHTpU6swAAHi0lSuls2el+vWltm2tTgMAgFsp2m5rC1WrVk3e3t559sInJCTk2VsvSSkpKdqxY4d27typUaNGSZKysrJkGIZ8fHz05Zdfqlu3bnm28/f3l7+/v3NeBAAAnih7in1UlJTPP8ADAICSc/k9835+foqIiNCaNWtyLV+zZo3at2+fZ/3g4GD99NNP+v7773MuI0eOVLNmzfT999+rTZs25RUdAADPlZIiffaZOWaKPQAAZc7l98xL0pgxY3TfffepdevWateunf7zn/8oLi5OI0eOlGROkT9y5Ig+/PBDeXl5qWXLlrm2DwsLU0BAQJ7lAADASZYvly5elK65RrrxRqvTAADgdmxR5qOionTq1ClNmTJFx44dU8uWLbVy5UrVr19fknTs2LGrnnMeAACUo+wp9kOGMMUeAAAnsMV55q3AeeYBACih06fN88qnp0u7d0vXXWd1IgAAbMGtzjMPAABsZulSs8hffz1FHgAAJ6HMAwCAsnX5FHsAAOAUlHkAAFB2jh+X/vtfcxwVZW0WAADcGGUeAACUnUWLpKws6eabpUaNrE4DAIDboswDAICywxR7AADKBWUeAACUjUOHpM2bzVPRDR5sdRoAANwaZR4AAJSNBQvM606dpNq1rc0CAICbo8wDAICywRR7AADKDWUeAACU3m+/STt2SN7e0oABVqcBAMDtUeYBAEDpzZ9vXnfvLoWFWZsFAAAPQJkHAAClN3eueX333dbmAADAQ1DmAQBA6ezaJe3eLfn5Sf37W50GAACPQJkHAAClk33gu969pcqVLY0CAICnoMwDAICSMwyOYg8AgAUo8wAAoORiY6Xff5cqVpRuv93qNAAAeAzKPAAAKLnsvfK33y4FBlqbBQAAD0KZBwAAJZOVdemUdEyxBwCgXFHmAQBAyWzdKh0+LAUHS7feanUaAAA8CmUeAACUTPYU+zvvlAICrM0CAICHocwDAIDiy8iQFi40x0yxBwCg3FHmAQBA8a1fLyUkSFWrSt27W50GAACPQ5kHAADFlz3FfuBAydfX2iwAAHggyjwAACietDRp8WJzzBR7AAAsQZkHAADF8+WXUmKiVLOm1KmT1WkAAPBIlHkAAFA82VPsBw+WvL2tzQIAgIeizAMAgKI7f1769FNzzBR7AAAsQ5kHAABFt3KldPasVL++1KaN1WkAAPBYlHkAAFB02VPshwyRHA5rswAA4MEo8wAAoGiSk6XPPzfHTLEHAMBSlHkAAFA0y5dLFy9KzZpJN9xgdRoAADwaZR4AABQNU+wBAHAZlHkAAHB1p09Lq1eb46goa7MAAADKPAAAKIIlS6SMDHN6ffPmVqcBAMDjUeYBAMDVXT7FHgAAWI4yDwAAChcfL61bZ46ZYg8AgEugzAMAgMItWiRlZUlt2kgNG1qdBgAAiDIPAACuhin2AAC4HMo8AAAoWFyctGWLeSq6QYOsTgMAAP6HMg8AAAq2YIF5fcstUu3a1mYBAAA5KPMAAKBgTLEHAMAlUeYBAED+9u2TYmMlb29pwACr0wAAgMtQ5gEAQP7mzzeve/SQqle3NgsAAMiFMg8AAPLHFHsAAFwWZR4AAOS1a5e0e7fk5yf17291GgAAcAXKPAAAyCt7r3zv3lLlypZGAQAAeVHmAQBAbobBFHsAAFwcZR4AAOQWGyv9/rtUsaJ0++1WpwEAAPmgzAMAgNyy98rffrsUGGhtFgAAkC/KPAAAuCQr69Ip6ZhiDwCAy6LMAwCAS7ZskQ4floKDpVtvtToNAAAoAGUeAABckj3F/s47pYAAa7MAAIACUeYBAIApI0NauNAcM8UeAACXRpkHAACmdeukEyekqlWl7t2tTgMAAApBmQcAAKbsKfYDB0q+vtZmAQAAhaLMAwAAKTVVWrLEHDPFHgAAl0eZBwAA0pdfSomJUs2aUqdOVqcBAABXQZkHAACXptgPHix5e1ubBQAAXBVlHgAAT3f+vPTpp+aYKfYAANgCZR4AAE/3+efSuXNS/fpSmzZWpwEAAEVAmQcAwNNlT7EfMkRyOKzNAgAAioQyDwCAJ0tONvfMS0yxBwDARijzAAB4sk8/NU9Ld+210g03WJ0GAAAUEWUeAABPxhR7AABsiTIPAICnOnXKPL+8JEVFWZsFAAAUC2UeAABPtWSJlJEh3XijOc0eAADYBmUeAABP9ckn5jUHvgMAwHYo8wAAeKLNm6X16yVvb8o8AAA2RJkHAMDTGIY0dqw5fughqX59a/MAAIBio8wDAOBpVq6UtmyRAgKkF16wOg0AACgByjwAAJ4kK0saN84cP/64VLu2tXkAAECJUOYBAPAkc+dKP/0khYRIzz1ndRoAAFBClHkAADxFWpo0YYI5fu45KTTU2jwAAKDEKPMAAHiK996TDhyQatQwp9gDAADboswDAOAJzp2TpkwxxxMmSIGB1uYBAAClQpkHAMATvPmmdPy41KiR9PDDVqcBAAClRJkHAMDdnTolTZ1qjv/+d8nPz9o8AACg1CjzAAC4u6lTpeRk6frrpSFDrE4DAADKAGUeAAB3duSI9K9/mePoaMmLX/0AALgDfqMDAODOpkyRLl6UOnaUeve2Og0AACgjlHkAANzVr79KMTHmODpacjiszQMAAMoMZR4AAHc1YYKUmSnddpu5Zx4AALgNyjwAAO4oNlZasMDcG//SS1anAQAAZYwyDwCAOxo/3ry+917zKPYAAMCtUOYBAHA369ZJX34p+fhIkydbnQYAADgBZR4AAHdiGNK4ceZ4xAipUSNr8wAAAKegzAMA4E4+/VTavl2qWFH629+sTgMAAJyEMg8AgLvIzLz0Xfknn5Rq1LA2DwAAcBrKPAAA7uKjj6S9e6XQUOmZZ6xOAwAAnIgyDwCAO0hNlSZONMfjxkkhIdbmAQAATkWZBwDAHbzzjhQXJ9WuLT36qNVpAACAk1HmAQCwu5QU6cUXzfHEiVKFCtbmAQAATkeZBwDA7l57TTp5UrrmGumBB6xOAwAAygFlHgAAOztxQnr1VXP84ouSj4+1eQAAQLmgzAMAYGcvvyydPStFREgDBlidBgAAlBPKPAAAdnXwoDR9ujl++WXJi1/rAAB4Cn7rAwBgV5MnS2lpUteuUs+eVqcBAADliDIPAIAd7dkjffCBOY6OlhwOa/MAAIByRZkHAMCO/vY3KStLuvNOqU0bq9MAAIByRpkHAMButm+Xli41vyOffX55AADgUSjzAADYiWFIY8ea42HDpOuuszYPAACwBGUeAAA7WbNGWr9e8vOTJk2yOg0AALCIbcr89OnT1bBhQwUEBCgiIkKbNm0qcN0lS5aoZ8+eql69uoKDg9WuXTutXr26HNMCAOAEWVnS+PHm+JFHpHr1rM0DAAAsY4syP3/+fI0ePVrPP/+8du7cqU6dOql3796Ki4vLd/2NGzeqZ8+eWrlypWJjY9W1a1fdfvvt2rlzZzknBwCgDC1eLMXGSkFBl0o9AADwSA7DMAyrQ1xNmzZt1KpVK82YMSNnWfPmzdW/f39FR0cX6TFatGihqKgovfDCC0VaPzk5WSEhIUpKSlJwcHCJcgMAUGbS06UWLaR9+8zp9RMnWp0IAACUseL0UJffM5+WlqbY2FhFRkbmWh4ZGamtW7cW6TGysrKUkpKi0NDQAtdJTU1VcnJyrgsAAC5j9myzyFerJo0ZY3UaAABgMZcv8ydPnlRmZqbCw8NzLQ8PD1d8fHyRHuOf//ynzp07p8GDBxe4TnR0tEJCQnIudevWLVVuAADKzIULlw529/zzUqVKlsYBAADWc/kyn83hcOS6bRhGnmX5mTt3riZNmqT58+crLCyswPXGjRunpKSknMuhQ4dKnRkAgDLx739LR4+aB7wbOdLqNAAAwAX4WB3gaqpVqyZvb+88e+ETEhLy7K2/0vz58/XQQw9p4cKF6tGjR6Hr+vv7y9/fv9R5AQAoU4mJUvbxYSZPlgICLI0DAABcg8vvmffz81NERITWrFmTa/maNWvUvn37ArebO3eu7r//fn3yySfq27evs2MCAOAcr74qnTkjXXeddN99VqcBAAAuwuX3zEvSmDFjdN9996l169Zq166d/vOf/yguLk4j/zfVcNy4cTpy5Ig+/PBDSWaRHzp0qN588021bds2Z69+hQoVFBISYtnrAACgWOLjpddfN8cvvSR5e1ubBwAAuAxblPmoqCidOnVKU6ZM0bFjx9SyZUutXLlS9evXlyQdO3Ys1znnZ86cqYyMDD366KN69NFHc5YPGzZMs2fPLu/4AACUzIsvSufPS23aSP36WZ0GAAC4EFucZ94KnGceAGCp/fulZs2kjAzpv/+Vuna1OhEAAHAytzrPPAAAHumFF8wiHxlJkQcAAHlQ5gEAcDU//ih98ok5fvlla7MAAACXRJkHAMDVPP+8ZBjS4MFSRITVaQAAgAuizAMA4Eo2b5Y++8w8cv3f/251GgAA4KIo8wAAuArDkMaONccPPihdc421eQAAgMuizAMA4CpWrpS2bJECAswD4AEAABSAMg8AgCvIypLGjzfHjz0m1aljbR4AAODSKPMAALiCefPMo9iHhFyaag8AAFAAyjwAAFZLS5MmTDDHzz4rhYZamwcAALg8yjwAAFZ77z1p/34pPFx64gmr0wAAABugzAMAYKVz56QpU8zxhAlSYKC1eQAAgC1Q5gEAsNKbb0rHj0sNG0rDh1udBgAA2ARlHgAAq5w+LU2bZo7//nfJz8/aPAAAwDYo8wAAWGXqVCkpSbr+eunuu61OAwAAbIQyDwCAFY4ckd56yxy//LLkxa9kAABQdPzlAACAFaZMkS5elDp0kPr0sToNAACwGco8AADl7ddfpZgYc/zKK5LDYW0eAABgO5R5AADK24QJUmam1Lev1LGj1WkAAIANUeYBAChP330nLVhg7o1/+WWr0wAAAJuizAMAUJ7Gjzev77nHPIo9AABACVDmAQAoL+vWSatXSz4+0uTJVqcBAAA2RpkHAKA8GIY0bpw5/utfpcaNrc0DAABsjTIPAEB5+PRTaft2qWJF6W9/szoNAACwOco8AADOlpl56bvyo0dLNWtaGgcAANgfZR4AAGf76CNp716pShXpmWesTgMAANwAZR4AAGdKTZUmTjTH48ZJlStbGgcAALgHyjwAAM70zjtSXJxUq5Y0apTVaQAAgJugzAMA4CwpKdKLL5rjiROlChWszQMAANwGZR4AAGd57TXp5EmpaVPpgQesTgMAANwIZR4AAGc4cUJ69VVz/OKLkq+vtXkAAIBbocwDAOAML78snT0rtWolDRxodRoAAOBmKPMAAJS1uDhp+nRz/PLLkhe/bgEAQNnirwsAAMrapElSWprUpYsUGWl1GgAA4IYo8wAAlKU9e6QPPjDH0dGSw2FtHgAA4JYo8wAAlKW//U3KypL695fatrU6DQAAcFOUeQAAysr27dLSpeZ35LPPLw8AAOAElHkAAMqCYUjjxpnj++6TWrSwNg8AAHBrlHkAAMrCu+9K69ZJfn7mAfAAAACciDIPAEBpGIb00kvSiBHm7aeflho0sDQSAABwfz5WBwAAwLYyM6XRo6V//9u8PW4c35UHAADlgjIPAEBJpKaa341fuNA8/dwbb0iPP251KgAA4CEo8wAAFFdSknTnneZ35H19pY8+kqKirE4FAAA8CGUeAIDiiI+XeveWvv9eqlTJPBVd9+5WpwIAAB6GMg8AQFHt2yf16iUdOCCFhUlffCG1amV1KgAA4IE4mj0AAEWxY4fUoYNZ5Bs3lrZupcgDAADLUOYBALiaL7+UunSRTpwwC/yWLWahBwAAsAhlHgCAwnz8sdS3r3TunNSjh7R+vRQebnUqAADg4SjzAAAU5LXXpL/8RcrIkIYMkT7/3DzoHQAAgMUo8wAAXCkrS3r2Wempp8zbo0ebe+j9/CyNBQAAkI2j2QMAcLn0dOmhh8xzx0vS1KnSM89IDoe1uQAAAC5DmQcAINvZs9KgQdKqVZK3txQTIw0bZnUqAACAPCjzAABI0smT5oHuvvlGqlhRWrhQ6tPH6lQAAAD5oswDAPDHH1KvXtKvv0qhoeaB7tq2tToVAABAgSjzAADP9uOP0q23SseOSfXqSatXS9dea3UqAACAQnE0ewCA59qwQerUySzyLVtKW7dS5AEAgC1Q5gEAnmnxYnNqfXKyWeg3bZJq17Y6FQAAQJFQ5gEAnmfGDPOo9amp0p13mlPrK1e2OhUAAECRUeYBAJ7DMKQXXpAeecQcjxhhHrW+QgWrkwEAABQLB8ADAHiGjAyzxL/7rnl70iSz2DsclsYCAAAoCco8AMD9Xbgg3X239OmnkpeX9Pbb0siRVqcCAAAoMco8AMC9nTkj3XGHtHmz5O8vzZ1rfk8eAADAxijzAAD3dfiweQ753bulkBBp+XLpllusTgUAAFBqlHkAgHvau9c89dyhQ1KtWtKqVdKf/mR1KgAAgDLB0ewBAO5n2zapY0ezyDdrJm3dSpEHAABuhTIPAHAvn30mde8unT4ttWljfle+fn2rUwEAAJQpyjwAwH3MmiX1728evb5PH+mrr6Rq1axOBQAAUOYo8wAA+zMMKTpaevBBKTNTGjZMWrZMCgy0OhkAAIBTUOYBAPaWlSWNHi2NH2/efu45cw+9r6+lsQAAAJyJo9kDAOwrNdXcCz9/vnn79dfNYg8AAODmKPMAAHtKTpbuusv8Xryvr/TBB9Ldd1udCgAAoFxQ5gEA9hMfbx7gbudOKShIWrJE6tnT6lQAAADlhjIPALCX336TevWS9u+XwsKklSuliAirUwEAAJQrDoAHALCP2FipQwezyDdqJG3ZQpEHAAAeiTIPALCHNWukLl2khATpppukrVulJk2sTgUAAGAJyjwAwPXNmyf17SudPSt17y6tXy+Fh1udCgAAwDKUeQCAa3vjDfMo9enpUlSU9PnnUnCw1akAAAAsRZkHALgmw5DGjpWefNK8/fjj0iefSP7+1uYCAABwARzNHgDgWtLSzL3v77wjffmluSw6WnruOcnhsDYbAACAi6DMAwCsZxjmkeo/+ECaO1c6dcpc7u0tvfuu9MAD1uYDAABwMZR5AIB1jhyR5syRPvxQ2rPn0vJataS//MUs8ddea10+AAAAF0WZBwCUr/PnpWXLzL3wa9dKWVnm8oAA6c47pWHDpB49zL3yAAAAyBdlHgDgfIYhbdpkFviFC6WUlEv3dexoFvhBg6SQEOsyAgAA2AhlHgDgPPv3m1PoP/xQOnDg0vKGDaWhQ6X77pMaN7YuHwAAgE1R5gEAZSspSVq0yNwLv2nTpeWVKpl734cNM/fGe3F2VAAAgJKizAMASi8z0/z++wcfSEuXShcvmssdDvP778OGmd+Hr1jR2pwAAABugjIPACi5PXvMAj9njnT06KXl115rFvi//EWqU8e6fAAAAG6KMg8AKJ6TJ6V588wSv2PHpeWhodLdd5slvnVrc688AAAAnIIyDwC4urQ0aeVKs8B//rmUnm4u9/GR+vQxC3zfvpK/v7U5AQAAPARlHgCQP8OQvvvOLPBz55p75LPddJNZ4O++WwoLsy4jAACAh6LMAwByO3rU/A78hx9Ku3dfWl6jhvkd+KFDpT/9ybp8AAAAoMwDACRduCAtW2buhV+zRsrKMpf7+0v9+5t74Xv2NKfVAwAAwHL8VQYAnsowpM2bzT3wCxZIycmX7mvf3izwgwdLlStbFhEAAAD5o8wDgKc5cMAs8B9+KO3ff2l5/frmFPqhQ6UmTazLBwAAgKuizAOAOzMM6fRpad8+6YcfpE8+kTZuvHR/YKA0aJC5F/6WWyQvL+uyAgAAoMgo8wBgd4ZhHmn+t9/M0v7bb7nHiYm513c4pG7dzAJ/111moQcAAICtUOYBwA4MQzpxIndJv7y4JyUVvn2dOubU+Z49pfvuk+rWLZ/cAAAAcArKPAC4CsOQEhLy37v+22+5D1CXn7p1zcLetGnu60aNpIoVy+c1AAAAoFxQ5gGgPBmGdPx4/nvXf/tNSkkpeFuHwyzs2SX9ysJeoUL5vQ4AAABYijIPAGXNMKT4+IIL+9mzBW/rcJhHlb+yrDdtKjVsKAUElN/rAAAAgMuizANASWRkmHvYr5wOn319/nzB23p5XSrsV+5lb9hQ8vcvv9cBAAAAW6LMA/AcWVnmXvHkZPOSkpL7uqjLUlIKL+uSWdgbNMj/O+wNGlDYAQAAUCq2KfPTp0/XP/7xDx07dkwtWrTQG2+8oU6dOhW4/oYNGzRmzBjt3r1btWrV0rPPPquRI0eWY2IAZcIwpHPnSle+s8eFTW8vCW9vs5jn9x32Bg0kP7+yfT4AAADgf2xR5ufPn6/Ro0dr+vTp6tChg2bOnKnevXtrz549qlevXp71Dxw4oD59+mj48OGaM2eOtmzZokceeUTVq1fXgAEDLHgFgAvLzJTS04t+SUsr3vrFedyzZ/PfC24YZfuafXyk4OBLl0qVcl/ntyy/+0JCzMcCAAAAypnDMMr6r+Sy16ZNG7Vq1UozZszIWda8eXP1799f0dHRedZ/7rnntHz5cu3duzdn2ciRI/XDDz9o27ZtRXrO5ORkhYSEKCkpScHBwaV/Ec5y8KC0Y8el21e+nfm9vVdbp6xvF7RO9rKSjku7fUmeLysr9yW/ZUW9v6T3FXXbjIyilWjX/1+AycureOW7sGX+/uaB5gAAAAAXUpwe6vK7lNLS0hQbG6uxY8fmWh4ZGamtW7fmu822bdsUGRmZa1mvXr0UExOj9PR0+fr65tkmNTVVqampObeTr3Y+Z1exYYM0bJjVKeBufH1LdvHzK/m2vr5SUFDBhbxCBQo4AAAA8D8uX+ZPnjypzMxMhYeH51oeHh6u+Pj4fLeJj4/Pd/2MjAydPHlSNWvWzLNNdHS0Jk+eXHbBy0tYmNSxY+5lVxaewm4XZ92y3Db7dvayko5Lu31xxl5euS/5LSvKfeW1rbd3yQq1tzelGQAAAHBxLl/mszmuKBeGYeRZdrX181uebdy4cRozZkzO7eTkZNWtW7ekccvPrbeaFwAAAACAx3D5Ml+tWjV5e3vn2QufkJCQZ+97tho1auS7vo+Pj6pWrZrvNv7+/vLnVFEAAAAAABvwsjrA1fj5+SkiIkJr1qzJtXzNmjVq3759vtu0a9cuz/pffvmlWrdune/35QEAAAAAsBOXL/OSNGbMGL333nt6//33tXfvXj355JOKi4vLOW/8uHHjNHTo0Jz1R44cqYMHD2rMmDHau3ev3n//fcXExOjpp5+26iUAAAAAAFBmXH6avSRFRUXp1KlTmjJlio4dO6aWLVtq5cqVql+/viTp2LFjiouLy1m/YcOGWrlypZ588km9/fbbqlWrlt566y3OMQ8AAAAAcAu2OM+8FWxznnkAAAAAgFsoTg+1xTR7AAAAAABwCWUeAAAAAACbocwDAAAAAGAzlHkAAAAAAGyGMg8AAAAAgM1Q5gEAAAAAsBnKPAAAAAAANkOZBwAAAADAZijzAAAAAADYDGUeAAAAAACbocwDAAAAAGAzlHkAAAAAAGyGMg8AAAAAgM1Q5gEAAAAAsBnKPAAAAAAANkOZBwAAAADAZijzAAAAAADYDGUeAAAAAACbocwDAAAAAGAzlHkAAAAAAGzGx+oArsowDElScnKyxUkAAAAAAJ4gu39m99HCUOYLkJKSIkmqW7euxUkAAAAAAJ4kJSVFISEhha7jMIpS+T1QVlaWjh49qkqVKsnhcFgdp1DJycmqW7euDh06pODgYKvjoBh47+yL986+eO/sjffPvnjv7Iv3zr547+zHMAylpKSoVq1a8vIq/Fvx7JkvgJeXl+rUqWN1jGIJDg7mQ2pTvHf2xXtnX7x39sb7Z1+8d/bFe2dfvHf2crU98tk4AB4AAAAAADZDmQcAAAAAwGYo827A399fEydOlL+/v9VRUEy8d/bFe2dfvHf2xvtnX7x39sV7Z1+8d+6NA+ABAAAAAGAz7JkHAAAAAMBmKPMAAAAAANgMZR4AAAAAAJuhzAMAAAAAYDOUeRt46aWX1L59e1WsWFGVK1fOd524uDjdfvvtCgwMVLVq1fT4448rLS2t0MdNTU3VY489pmrVqikwMFB33HGHDh8+7IRXgGzr16+Xw+HI9/Ltt98WuN3999+fZ/22bduWY3JIUoMGDfK8D2PHji10G8MwNGnSJNWqVUsVKlRQly5dtHv37nJKDEn6448/9NBDD6lhw4aqUKGCGjdurIkTJ171/5F87qwxffp0NWzYUAEBAYqIiNCmTZsKXX/Dhg2KiIhQQECAGjVqpHfeeaeckuJy0dHR+vOf/6xKlSopLCxM/fv31y+//FLoNgX9Tvz555/LKTUkadKkSXnegxo1ahS6DZ8715Df3yUOh0OPPvpovuvzmXM/PlYHwNWlpaVp0KBBateunWJiYvLcn5mZqb59+6p69eravHmzTp06pWHDhskwDP3rX/8q8HFHjx6tFStWaN68eapataqeeuop3XbbbYqNjZW3t7czX5LHat++vY4dO5Zr2YQJE7R27Vq1bt260G1vvfVWzZo1K+e2n5+fUzKicFOmTNHw4cNzbgcFBRW6/rRp0/Taa69p9uzZuuaaa/Tiiy+qZ8+e+uWXX1SpUiVnx4Wkn3/+WVlZWZo5c6aaNGmiXbt2afjw4Tp37pxeffXVQrflc1e+5s+fr9GjR2v69Onq0KGDZs6cqd69e2vPnj2qV69envUPHDigPn36aPjw4ZozZ462bNmiRx55RNWrV9eAAQMseAWea8OGDXr00Uf15z//WRkZGXr++ecVGRmpPXv2KDAwsNBtf/nlFwUHB+fcrl69urPj4gotWrTQ2rVrc24X9ncgnzvX8e233yozMzPn9q5du9SzZ08NGjSo0O34zLkRA7Yxa9YsIyQkJM/ylStXGl5eXsaRI0dyls2dO9fw9/c3kpKS8n2sxMREw9fX15g3b17OsiNHjhheXl7GqlWryjw78peWlmaEhYUZU6ZMKXS9YcOGGf369SufUChQ/fr1jddff73I62dlZRk1atQwXnnllZxlFy9eNEJCQox33nnHCQlRVNOmTTMaNmxY6Dp87srfzTffbIwcOTLXsmuvvdYYO3Zsvus/++yzxrXXXptr2YgRI4y2bds6LSOKJiEhwZBkbNiwocB11q1bZ0gyzpw5U37BkMfEiRONG264ocjr87lzXU888YTRuHFjIysrK9/7+cy5H6bZu4Ft27apZcuWqlWrVs6yXr16KTU1VbGxsfluExsbq/T0dEVGRuYsq1Wrllq2bKmtW7c6PTNMy5cv18mTJ3X//fdfdd3169crLCxM11xzjYYPH66EhATnB0QeU6dOVdWqVXXjjTfqpZdeKnSq9oEDBxQfH5/rc+bv76/OnTvzObNYUlKSQkNDr7oen7vyk5aWptjY2FyfF0mKjIws8POybdu2POv36tVLO3bsUHp6utOy4uqSkpIkqUifs5tuukk1a9ZU9+7dtW7dOmdHQz727dunWrVqqWHDhhoyZIj2799f4Lp87lxTWlqa5syZowcffFAOh6PQdfnMuQ/KvBuIj49XeHh4rmVVqlSRn5+f4uPjC9zGz89PVapUybU8PDy8wG1Q9mJiYtSrVy/VrVu30PV69+6tjz/+WP/973/1z3/+U99++626deum1NTUckoKSXriiSc0b948rVu3TqNGjdIbb7yhRx55pMD1sz9LV34++ZxZ6/fff9e//vUvjRw5stD1+NyVr5MnTyozM7NYn5f8fv+Fh4crIyNDJ0+edFpWFM4wDI0ZM0YdO3ZUy5YtC1yvZs2a+s9//qPFixdryZIlatasmbp3766NGzeWY1q0adNGH374oVavXq13331X8fHxat++vU6dOpXv+nzuXNOyZcuUmJhY6A4iPnPuh+/MW2TSpEmaPHlyoet8++23V/0edbb8/gXOMIyr/stcWWyDkr2fhw8f1urVq7VgwYKrPn5UVFTOuGXLlmrdurXq16+vzz//XHfddVfJg6NY792TTz6Zs+z6669XlSpVNHDgwJy99QW58jPF56xslORzd/ToUd16660aNGiQHn744UK35XNnjeJ+XvJbP7/lKD+jRo3Sjz/+qM2bNxe6XrNmzdSsWbOc2+3atdOhQ4f06quv6pZbbnF2TPxP7969c8Z/+tOf1K5dOzVu3FgffPCBxowZk+82fO5cT0xMjHr37p1rpu6V+My5H8q8RUaNGqUhQ4YUuk6DBg2K9Fg1atTQ9u3bcy07c+aM0tPT8/zL6eXbpKWl6cyZM7n2zickJKh9+/ZFel5cUpL3c9asWapataruuOOOYj9fzZo1Vb9+fe3bt6/Y2yK30nwWs49s/ttvv+Vb5rOPBhwfH6+aNWvmLE9ISCjws4miK+57d/ToUXXt2lXt2rXTf/7zn2I/H58756pWrZq8vb3z7IUv7PNSo0aNfNf38fEp9B/Y4DyPPfaYli9fro0bN6pOnTrF3r5t27aaM2eOE5KhqAIDA/WnP/2pwP/X8blzPQcPHtTatWu1ZMmSYm/LZ87eKPMWqVatmqpVq1Ymj9WuXTu99NJLOnbsWE5h+PLLL+Xv76+IiIh8t4mIiJCvr6/WrFmjwYMHS5KOHTumXbt2adq0aWWSy5MU9/00DEOzZs3S0KFD5evrW+znO3XqlA4dOpSrIKJkSvNZ3LlzpyQV+D40bNhQNWrU0Jo1a3TTTTdJMr/TtmHDBk2dOrVkgZGjOO/dkSNH1LVrV0VERGjWrFny8ir+t8z43DmXn5+fIiIitGbNGt155505y9esWaN+/frlu027du20YsWKXMu+/PJLtW7dukT/b0XJGYahxx57TEuXLtX69evVsGHDEj3Ozp07+YxZLDU1VXv37lWnTp3yvZ/PneuZNWuWwsLC1Ldv32Jvy2fO5iw79B6K7ODBg8bOnTuNyZMnG0FBQcbOnTuNnTt3GikpKYZhGEZGRobRsmVLo3v37sZ3331nrF271qhTp44xatSonMc4fPiw0axZM2P79u05y0aOHGnUqVPHWLt2rfHdd98Z3bp1M2644QYjIyOj3F+jp1m7dq0hydizZ0++9zdr1sxYsmSJYRiGkZKSYjz11FPG1q1bjQMHDhjr1q0z2rVrZ9SuXdtITk4uz9gebevWrcZrr71m7Ny509i/f78xf/58o1atWsYdd9yRa73L3zvDMIxXXnnFCAkJMZYsWWL89NNPxt13323UrFmT964cHTlyxGjSpInRrVs34/Dhw8axY8dyLpfjc2e9efPmGb6+vkZMTIyxZ88eY/To0UZgYKDxxx9/GIZhGGPHjjXuu+++nPX3799vVKxY0XjyySeNPXv2GDExMYavr6+xaNEiq16Cx/q///s/IyQkxFi/fn2uz9j58+dz1rny/Xv99deNpUuXGr/++quxa9cuY+zYsYYkY/HixVa8BI/11FNPGevXrzf2799vfP3118Ztt91mVKpUic+dTWRmZhr16tUznnvuuTz38Zlzf5R5Gxg2bJghKc9l3bp1OescPHjQ6Nu3r1GhQgUjNDTUGDVqlHHx4sWc+w8cOJBnmwsXLhijRo0yQkNDjQoVKhi33XabERcXV46vzHPdfffdRvv27Qu8X5Ixa9YswzAM4/z580ZkZKRRvXp1w9fX16hXr54xbNgw3qtyFhsba7Rp08YICQkxAgICjGbNmhkTJ040zp07l2u9y987wzBPTzdx4kSjRo0ahr+/v3HLLbcYP/30Uzmn92yzZs3K9/+hV/57Np871/D2228b9evXN/z8/IxWrVrlOrXZsGHDjM6dO+daf/369cZNN91k+Pn5GQ0aNDBmzJhRzolhGEaBn7HL/3945fs3depUo3HjxkZAQIBRpUoVo2PHjsbnn39e/uE9XFRUlFGzZk3D19fXqFWrlnHXXXcZu3fvzrmfz51rW716tSHJ+OWXX/Lcx2fO/TkM439HrAAAAAAAALbAqekAAAAAALAZyjwAAAAAADZDmQcAAAAAwGYo8wAAAAAA2AxlHgAAAAAAm6HMAwAAAABgM5R5AAAAAABshjIPAAAAAIDNUOYBAAAAALAZyjwAAAAAADZDmQcAAAAAwGYo8wAAAAAA2AxlHgAAAAAAm6HMAwCAUpk0aZIcDkeRLn/88YfVcQEAcAs+VgcAAAD2FhQUpPDw8ALvP3/+vFJSUsoxEQAA7s9hGIZhdQgAAOCekpKS1K5dO+3du1etWrXS1q1b5e/vb3UsAABsj2n2AADAKTIyMjRgwADt3btXtWvX1vLlyynyAACUEco8AABwiv/7v//TV199pcDAQC1fvly1a9e2OhIAAG6DMg8AAMrc1KlT9d5778nLy0tz5sxRq1atrI4EAIBbocwDAIAytXjxYo0bN06SWer79+9vbSAAANwQB8ADAABl5ptvvlGXLl104cIFPfzww3r33XetjgQAgFuizAMAgDJx8OBBtWnTRsePH1fXrl21evVq+fr6Wh0LAAC3RJkHAACllpycrA4dOmjXrl265ppr9PXXX6tKlSpWxwIAwG3xnXkAAFAqGRkZGjx4sHbt2qWqVavq888/p8gDAOBklHkAAFAqTzzxhFavXi0/Pz8tXrxYTZo0sToSAABuj2n2AACgVBo0aKCDBw/K19dXoaGhha777bffqm7duuWUDAAA9+VjdQAAAOAe0tPTdfz48ULXyczMLKc0AAC4N/bMAwAAAABgM3xnHgAAAAAAm6HMAwAAAABgM5R5AAAAAABshjIPAAAAAIDNUOYBAAAAALAZyjwAAAAAADZDmQcAAAAAwGYo8wAAAAAA2AxlHgAAAAAAm6HMAwAAAABgM5R5AAAAAABshjIPAAAAAIDNUOYBAAAAALCZ/wcw6KFoHHPTXAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1200x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))\n",
    "\n",
    "\n",
    "nums = np.arange(-10, 10, step=1)\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(12, 8))\n",
    "ax.plot(nums, sigmoid(nums), 'r')\n",
    "ax.set_xlabel('z', fontsize=18)\n",
    "ax.set_ylabel('g(z)', fontsize=18)\n",
    "ax.set_title('sigmoid function', fontsize=18)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "apEHwiQmvoH_"
   },
   "source": [
    "## Cost Function 代价函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Zw3ZmdqVjlII"
   },
   "source": [
    "棒极了！现在，我们需要编写代价函数来评估结果。\n",
    "代价函数：\n",
    "$J\\left( \\theta  \\right)=\\frac{1}{m}\\sum\\limits_{i=1}^{m}{[-{{y}^{(i)}}\\log \\left( {{h}_{\\theta }}\\left( {{x}^{(i)}} \\right) \\right)-\\left( 1-{{y}^{(i)}} \\right)\\log \\left( 1-{{h}_{\\theta }}\\left( {{x}^{(i)}} \\right) \\right)]}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "R0Y4hVpkjlIJ"
   },
   "outputs": [],
   "source": [
    "def cost(theta, X, y):\n",
    "    return np.mean(-y * np.log(sigmoid(X @ theta)) - (1 - y) * np.log(1 - sigmoid(X @ theta)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XEBSMFxujlIS"
   },
   "source": [
    "让我们来检查矩阵的维度来确保一切良好。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "6Ax8oM2AjlIT"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0., 0., 0.])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta = np.zeros(3)\n",
    "theta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8OY_-JQ8jlIX"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((3, 1), (3,), (3, 1))"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape, theta.shape, y.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ZZxmEmkwjlIa"
   },
   "source": [
    "让我们计算初始化参数的代价函数(theta为0)。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "fXdMxsdrjlIb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7.675283892942822\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "\n",
    "def sigmoid(x):\n",
    "    return 1 / (1 + np.exp(-x))\n",
    "\n",
    "\n",
    "def cost(theta, X, y):\n",
    "    eps = 1e-10  # 定义一个小的偏移量\n",
    "    m = len(y)\n",
    "    h = sigmoid(X @ theta)\n",
    "    # 避免除以零的情况\n",
    "    h = np.clip(h, eps, 1 - eps)\n",
    "    J = np.mean(-y * np.log(h) - (1 - y) * np.log(1 - h))\n",
    "    return J\n",
    "\n",
    "\n",
    "theta = np.array([1, 2, 3]).reshape(-1, 1)\n",
    "X = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])\n",
    "y = np.array([1, 0, 1]).reshape(-1, 1)\n",
    "print(cost(theta, X, y))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "SSGkYiVXjlIf"
   },
   "source": [
    "看起来不错，接下来，我们需要一个函数来计算我们的训练数据、标签和一些参数thata的梯度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "tI-AX8knjlIi"
   },
   "source": [
    "## Gradient descent 梯度下降\n",
    "* 这是批量梯度下降（batch gradient descent）  \n",
    "* 转化为向量化计算： $\\frac{1}{m} X^T( Sigmoid(X\\theta) - y )$\n",
    "$$\\frac{\\partial J\\left( \\theta  \\right)}{\\partial {{\\theta }_{j}}}=\\frac{1}{m}\\sum\\limits_{i=1}^{m}{\\left({h_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}}\\right)x_{_{j}}^{(i)}}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "D1SA0lHXjlIj"
   },
   "outputs": [],
   "source": [
    "# Gradient的实现，利用的是循环的方法\n",
    "def gradient(theta, X, y):\n",
    "    theta = np.matrix(theta)\n",
    "    X = np.matrix(X)\n",
    "    y = np.matrix(y).T\n",
    "    print(\"X.shape = \", X.shape)\n",
    "    print(\"y.shape = \", y.shape)\n",
    "    print(\"theta.shape = \", theta.shape)\n",
    "    \n",
    "    parameters = int(theta.ravel().shape[1])\n",
    "    grad = np.zeros(parameters)\n",
    "    \n",
    "    error = sigmoid(X @ theta.T) - y\n",
    "    print(\"error.shape = \", error.shape)\n",
    "    \n",
    "    for i in range(parameters):\n",
    "        term = X[:,i].T @ error\n",
    "        grad[i] = term / len(X)\n",
    "    \n",
    "    return grad"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "brjL4gt34jMb"
   },
   "outputs": [],
   "source": [
    "# 另外一种实现的方法\n",
    "def gradient(theta, X, y):\n",
    "    return (1 / len(X)) * X.T @ (sigmoid(X @ theta) - y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6TEJDxaqjlIr"
   },
   "source": [
    "注意，我们实际上没有在这个函数中执行梯度下降，我们仅仅在计算一个梯度步长。在练习中，一个称为“fminunc”的Octave函数是用来优化函数来计算成本和梯度参数。由于我们使用Python，我们可以用SciPy的“optimize”命名空间来做同样的事情。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "FoLQqZg7jlIt"
   },
   "source": [
    "我们看看用我们的数据和初始参数为0的梯度下降法的结果。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "twQhHqydjlIw"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.33333306],\n",
       "       [1.66666611],\n",
       "       [1.99999917]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gradient(theta, X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "M5cP5VjE-YdI"
   },
   "source": [
    "## 拟合参数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3ye-zhoBjlI1"
   },
   "source": [
    "现在可以用SciPy's truncated newton（TNC）实现寻找最优参数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "0UHRpG7ijlI2"
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "shapes (3,4) and (3,) not aligned: 4 (dim 1) != 3 (dim 0)",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[27], line 2\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mscipy\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01moptimize\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01mopt\u001b[39;00m\n\u001b[1;32m----> 2\u001b[0m res \u001b[38;5;241m=\u001b[39m opt\u001b[38;5;241m.\u001b[39mminimize(fun\u001b[38;5;241m=\u001b[39mcost, x0\u001b[38;5;241m=\u001b[39mtheta, args\u001b[38;5;241m=\u001b[39m(X, y), method\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mNewton-CG\u001b[39m\u001b[38;5;124m'\u001b[39m, jac\u001b[38;5;241m=\u001b[39mgradient)\n\u001b[0;32m      3\u001b[0m \u001b[38;5;28mprint\u001b[39m(res)\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_minimize.py:710\u001b[0m, in \u001b[0;36mminimize\u001b[1;34m(fun, x0, args, method, jac, hess, hessp, bounds, constraints, tol, callback, options)\u001b[0m\n\u001b[0;32m    708\u001b[0m     res \u001b[38;5;241m=\u001b[39m _minimize_bfgs(fun, x0, args, jac, callback, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions)\n\u001b[0;32m    709\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m meth \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mnewton-cg\u001b[39m\u001b[38;5;124m'\u001b[39m:\n\u001b[1;32m--> 710\u001b[0m     res \u001b[38;5;241m=\u001b[39m _minimize_newtoncg(fun, x0, args, jac, hess, hessp, callback,\n\u001b[0;32m    711\u001b[0m                              \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions)\n\u001b[0;32m    712\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m meth \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124ml-bfgs-b\u001b[39m\u001b[38;5;124m'\u001b[39m:\n\u001b[0;32m    713\u001b[0m     res \u001b[38;5;241m=\u001b[39m _minimize_lbfgsb(fun, x0, args, jac, bounds,\n\u001b[0;32m    714\u001b[0m                            callback\u001b[38;5;241m=\u001b[39mcallback, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions)\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_optimize.py:1992\u001b[0m, in \u001b[0;36m_minimize_newtoncg\u001b[1;34m(fun, x0, args, jac, hess, hessp, callback, xtol, eps, maxiter, disp, return_all, c1, c2, **unknown_options)\u001b[0m\n\u001b[0;32m   1990\u001b[0m x0 \u001b[38;5;241m=\u001b[39m asarray(x0)\u001b[38;5;241m.\u001b[39mflatten()\n\u001b[0;32m   1991\u001b[0m \u001b[38;5;66;03m# TODO: add hessp (callable or FD) to ScalarFunction?\u001b[39;00m\n\u001b[1;32m-> 1992\u001b[0m sf \u001b[38;5;241m=\u001b[39m _prepare_scalar_function(\n\u001b[0;32m   1993\u001b[0m     fun, x0, jac, args\u001b[38;5;241m=\u001b[39margs, epsilon\u001b[38;5;241m=\u001b[39meps, hess\u001b[38;5;241m=\u001b[39mhess\n\u001b[0;32m   1994\u001b[0m )\n\u001b[0;32m   1995\u001b[0m f \u001b[38;5;241m=\u001b[39m sf\u001b[38;5;241m.\u001b[39mfun\n\u001b[0;32m   1996\u001b[0m fprime \u001b[38;5;241m=\u001b[39m sf\u001b[38;5;241m.\u001b[39mgrad\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_optimize.py:288\u001b[0m, in \u001b[0;36m_prepare_scalar_function\u001b[1;34m(fun, x0, jac, args, bounds, epsilon, finite_diff_rel_step, hess)\u001b[0m\n\u001b[0;32m    284\u001b[0m     bounds \u001b[38;5;241m=\u001b[39m (\u001b[38;5;241m-\u001b[39mnp\u001b[38;5;241m.\u001b[39minf, np\u001b[38;5;241m.\u001b[39minf)\n\u001b[0;32m    286\u001b[0m \u001b[38;5;66;03m# ScalarFunction caches. Reuse of fun(x) during grad\u001b[39;00m\n\u001b[0;32m    287\u001b[0m \u001b[38;5;66;03m# calculation reduces overall function evaluations.\u001b[39;00m\n\u001b[1;32m--> 288\u001b[0m sf \u001b[38;5;241m=\u001b[39m ScalarFunction(fun, x0, args, grad, hess,\n\u001b[0;32m    289\u001b[0m                     finite_diff_rel_step, bounds, epsilon\u001b[38;5;241m=\u001b[39mepsilon)\n\u001b[0;32m    291\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m sf\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_differentiable_functions.py:166\u001b[0m, in \u001b[0;36mScalarFunction.__init__\u001b[1;34m(self, fun, x0, args, grad, hess, finite_diff_rel_step, finite_diff_bounds, epsilon)\u001b[0m\n\u001b[0;32m    163\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mf \u001b[38;5;241m=\u001b[39m fun_wrapped(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mx)\n\u001b[0;32m    165\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_update_fun_impl \u001b[38;5;241m=\u001b[39m update_fun\n\u001b[1;32m--> 166\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_update_fun()\n\u001b[0;32m    168\u001b[0m \u001b[38;5;66;03m# Gradient evaluation\u001b[39;00m\n\u001b[0;32m    169\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mcallable\u001b[39m(grad):\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_differentiable_functions.py:262\u001b[0m, in \u001b[0;36mScalarFunction._update_fun\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    260\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21m_update_fun\u001b[39m(\u001b[38;5;28mself\u001b[39m):\n\u001b[0;32m    261\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mf_updated:\n\u001b[1;32m--> 262\u001b[0m         \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39m_update_fun_impl()\n\u001b[0;32m    263\u001b[0m         \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mf_updated \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;01mTrue\u001b[39;00m\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_differentiable_functions.py:163\u001b[0m, in \u001b[0;36mScalarFunction.__init__.<locals>.update_fun\u001b[1;34m()\u001b[0m\n\u001b[0;32m    162\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mupdate_fun\u001b[39m():\n\u001b[1;32m--> 163\u001b[0m     \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mf \u001b[38;5;241m=\u001b[39m fun_wrapped(\u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mx)\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_differentiable_functions.py:145\u001b[0m, in \u001b[0;36mScalarFunction.__init__.<locals>.fun_wrapped\u001b[1;34m(x)\u001b[0m\n\u001b[0;32m    141\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mnfev \u001b[38;5;241m+\u001b[39m\u001b[38;5;241m=\u001b[39m \u001b[38;5;241m1\u001b[39m\n\u001b[0;32m    142\u001b[0m \u001b[38;5;66;03m# Send a copy because the user may overwrite it.\u001b[39;00m\n\u001b[0;32m    143\u001b[0m \u001b[38;5;66;03m# Overwriting results in undefined behaviour because\u001b[39;00m\n\u001b[0;32m    144\u001b[0m \u001b[38;5;66;03m# fun(self.x) will change self.x, with the two no longer linked.\u001b[39;00m\n\u001b[1;32m--> 145\u001b[0m fx \u001b[38;5;241m=\u001b[39m fun(np\u001b[38;5;241m.\u001b[39mcopy(x), \u001b[38;5;241m*\u001b[39margs)\n\u001b[0;32m    146\u001b[0m \u001b[38;5;66;03m# Make sure the function returns a true scalar\u001b[39;00m\n\u001b[0;32m    147\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m np\u001b[38;5;241m.\u001b[39misscalar(fx):\n",
      "Cell \u001b[1;32mIn[24], line 15\u001b[0m, in \u001b[0;36mcost\u001b[1;34m(theta, X, y)\u001b[0m\n\u001b[0;32m     13\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mcost\u001b[39m(theta, X, y):\n\u001b[0;32m     14\u001b[0m     m \u001b[38;5;241m=\u001b[39m \u001b[38;5;28mlen\u001b[39m(y)\n\u001b[1;32m---> 15\u001b[0m     h \u001b[38;5;241m=\u001b[39m sigmoid(np\u001b[38;5;241m.\u001b[39mdot(X, theta))\n\u001b[0;32m     16\u001b[0m     \u001b[38;5;66;03m# 确保 h 的值在 (0, 1) 内，避免 np.log 出现问题\u001b[39;00m\n\u001b[0;32m     17\u001b[0m     h \u001b[38;5;241m=\u001b[39m np\u001b[38;5;241m.\u001b[39mclip(h, \u001b[38;5;241m1e-10\u001b[39m, \u001b[38;5;241m1\u001b[39m \u001b[38;5;241m-\u001b[39m \u001b[38;5;241m1e-10\u001b[39m)\n",
      "\u001b[1;31mValueError\u001b[0m: shapes (3,4) and (3,) not aligned: 4 (dim 1) != 3 (dim 0)"
     ]
    }
   ],
   "source": [
    "import scipy.optimize as opt\n",
    "res = opt.minimize(fun=cost, x0=theta, args=(X, y), method='Newton-CG', jac=gradient)\n",
    "print(res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "o-z0iOvZjlI5"
   },
   "source": [
    "让我们看看在这个结论下代价函数计算结果是什么个样子~"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "d7Sc0NlzjlI6",
    "scrolled": true
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "operands could not be broadcast together with shapes (4,) (12,) ",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[34], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m res \u001b[38;5;241m=\u001b[39m opt\u001b[38;5;241m.\u001b[39mminimize(fun\u001b[38;5;241m=\u001b[39mcost, x0\u001b[38;5;241m=\u001b[39mtheta, args\u001b[38;5;241m=\u001b[39m(X, y), method\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mNewton-CG\u001b[39m\u001b[38;5;124m'\u001b[39m, jac\u001b[38;5;241m=\u001b[39mgradient)\n\u001b[0;32m      3\u001b[0m \u001b[38;5;66;03m# 再使用 res.x 调用 cost 函数\u001b[39;00m\n\u001b[0;32m      4\u001b[0m final_cost \u001b[38;5;241m=\u001b[39m cost(res\u001b[38;5;241m.\u001b[39mx, X, y)\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_minimize.py:710\u001b[0m, in \u001b[0;36mminimize\u001b[1;34m(fun, x0, args, method, jac, hess, hessp, bounds, constraints, tol, callback, options)\u001b[0m\n\u001b[0;32m    708\u001b[0m     res \u001b[38;5;241m=\u001b[39m _minimize_bfgs(fun, x0, args, jac, callback, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions)\n\u001b[0;32m    709\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m meth \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mnewton-cg\u001b[39m\u001b[38;5;124m'\u001b[39m:\n\u001b[1;32m--> 710\u001b[0m     res \u001b[38;5;241m=\u001b[39m _minimize_newtoncg(fun, x0, args, jac, hess, hessp, callback,\n\u001b[0;32m    711\u001b[0m                              \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions)\n\u001b[0;32m    712\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m meth \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124ml-bfgs-b\u001b[39m\u001b[38;5;124m'\u001b[39m:\n\u001b[0;32m    713\u001b[0m     res \u001b[38;5;241m=\u001b[39m _minimize_lbfgsb(fun, x0, args, jac, bounds,\n\u001b[0;32m    714\u001b[0m                            callback\u001b[38;5;241m=\u001b[39mcallback, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions)\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_optimize.py:2072\u001b[0m, in \u001b[0;36m_minimize_newtoncg\u001b[1;34m(fun, x0, args, jac, hess, hessp, callback, xtol, eps, maxiter, disp, return_all, c1, c2, **unknown_options)\u001b[0m\n\u001b[0;32m   2070\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m fhess \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[0;32m   2071\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m fhess_p \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m-> 2072\u001b[0m         Ap \u001b[38;5;241m=\u001b[39m approx_fhess_p(xk, psupi, fprime, epsilon)\n\u001b[0;32m   2073\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m   2074\u001b[0m         Ap \u001b[38;5;241m=\u001b[39m fhess_p(xk, psupi, \u001b[38;5;241m*\u001b[39margs)\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_optimize.py:1110\u001b[0m, in \u001b[0;36mapprox_fhess_p\u001b[1;34m(x0, p, fprime, epsilon, *args)\u001b[0m\n\u001b[0;32m   1107\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mapprox_fhess_p\u001b[39m(x0, p, fprime, epsilon, \u001b[38;5;241m*\u001b[39margs):\n\u001b[0;32m   1108\u001b[0m     \u001b[38;5;66;03m# calculate fprime(x0) first, as this may be cached by ScalarFunction\u001b[39;00m\n\u001b[0;32m   1109\u001b[0m     f1 \u001b[38;5;241m=\u001b[39m fprime(\u001b[38;5;241m*\u001b[39m((x0,) \u001b[38;5;241m+\u001b[39m args))\n\u001b[1;32m-> 1110\u001b[0m     f2 \u001b[38;5;241m=\u001b[39m fprime(\u001b[38;5;241m*\u001b[39m((x0 \u001b[38;5;241m+\u001b[39m epsilon\u001b[38;5;241m*\u001b[39mp,) \u001b[38;5;241m+\u001b[39m args))\n\u001b[0;32m   1111\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m (f2 \u001b[38;5;241m-\u001b[39m f1) \u001b[38;5;241m/\u001b[39m epsilon\n",
      "\u001b[1;31mValueError\u001b[0m: operands could not be broadcast together with shapes (4,) (12,) "
     ]
    }
   ],
   "source": [
    "cost(res.x, X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "jqbJJue1-lK6"
   },
   "source": [
    "## 用训练集预测和验证"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "F5xbkpoXjlJD"
   },
   "source": [
    "接下来，我们需要编写一个函数，用我们所学的参数theta来为数据集X输出预测。然后，我们可以使用这个函数来给我们的分类器的训练精度打分。\n",
    "逻辑回归模型的假设函数： \n",
    "\t\\\\[{{h}_{\\theta }}\\left( x \\right)=\\frac{1}{1+{{e}^{-{{\\theta }^{T}}X}}}\\\\] \n",
    "当${{h}_{\\theta }}$大于等于0.5时，预测 y=1\n",
    "\n",
    "当${{h}_{\\theta }}$小于0.5时，预测 y=0 。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "L5kMfIWijlJE"
   },
   "outputs": [],
   "source": [
    "def predict(theta, X):\n",
    "    probability = sigmoid(X @ theta)\n",
    "    return [1 if x >= 0.5 else 0 for x in probability]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "5yDGG6aXjlJI"
   },
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "operands could not be broadcast together with shapes (4,) (12,) ",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[35], line 42\u001b[0m\n\u001b[0;32m     39\u001b[0m X \u001b[38;5;241m=\u001b[39m np\u001b[38;5;241m.\u001b[39mcolumn_stack((np\u001b[38;5;241m.\u001b[39mones(\u001b[38;5;28mlen\u001b[39m(X)), X))\n\u001b[0;32m     41\u001b[0m \u001b[38;5;66;03m# 先使用 opt.minimize 函数得到 res\u001b[39;00m\n\u001b[1;32m---> 42\u001b[0m res \u001b[38;5;241m=\u001b[39m opt\u001b[38;5;241m.\u001b[39mminimize(fun\u001b[38;5;241m=\u001b[39mcost, x0\u001b[38;5;241m=\u001b[39mtheta, args\u001b[38;5;241m=\u001b[39m(X, y), method\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mNewton-CG\u001b[39m\u001b[38;5;124m'\u001b[39m, jac\u001b[38;5;241m=\u001b[39mgradient)\n\u001b[0;32m     44\u001b[0m theta_min \u001b[38;5;241m=\u001b[39m res\u001b[38;5;241m.\u001b[39mx\n\u001b[0;32m     45\u001b[0m predictions \u001b[38;5;241m=\u001b[39m predict(theta_min, X)\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_minimize.py:710\u001b[0m, in \u001b[0;36mminimize\u001b[1;34m(fun, x0, args, method, jac, hess, hessp, bounds, constraints, tol, callback, options)\u001b[0m\n\u001b[0;32m    708\u001b[0m     res \u001b[38;5;241m=\u001b[39m _minimize_bfgs(fun, x0, args, jac, callback, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions)\n\u001b[0;32m    709\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m meth \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mnewton-cg\u001b[39m\u001b[38;5;124m'\u001b[39m:\n\u001b[1;32m--> 710\u001b[0m     res \u001b[38;5;241m=\u001b[39m _minimize_newtoncg(fun, x0, args, jac, hess, hessp, callback,\n\u001b[0;32m    711\u001b[0m                              \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions)\n\u001b[0;32m    712\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m meth \u001b[38;5;241m==\u001b[39m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124ml-bfgs-b\u001b[39m\u001b[38;5;124m'\u001b[39m:\n\u001b[0;32m    713\u001b[0m     res \u001b[38;5;241m=\u001b[39m _minimize_lbfgsb(fun, x0, args, jac, bounds,\n\u001b[0;32m    714\u001b[0m                            callback\u001b[38;5;241m=\u001b[39mcallback, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39moptions)\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_optimize.py:2072\u001b[0m, in \u001b[0;36m_minimize_newtoncg\u001b[1;34m(fun, x0, args, jac, hess, hessp, callback, xtol, eps, maxiter, disp, return_all, c1, c2, **unknown_options)\u001b[0m\n\u001b[0;32m   2070\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m fhess \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[0;32m   2071\u001b[0m     \u001b[38;5;28;01mif\u001b[39;00m fhess_p \u001b[38;5;129;01mis\u001b[39;00m \u001b[38;5;28;01mNone\u001b[39;00m:\n\u001b[1;32m-> 2072\u001b[0m         Ap \u001b[38;5;241m=\u001b[39m approx_fhess_p(xk, psupi, fprime, epsilon)\n\u001b[0;32m   2073\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m   2074\u001b[0m         Ap \u001b[38;5;241m=\u001b[39m fhess_p(xk, psupi, \u001b[38;5;241m*\u001b[39margs)\n",
      "File \u001b[1;32mD:\\ProgramData\\anaconda3\\Lib\\site-packages\\scipy\\optimize\\_optimize.py:1110\u001b[0m, in \u001b[0;36mapprox_fhess_p\u001b[1;34m(x0, p, fprime, epsilon, *args)\u001b[0m\n\u001b[0;32m   1107\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21mapprox_fhess_p\u001b[39m(x0, p, fprime, epsilon, \u001b[38;5;241m*\u001b[39margs):\n\u001b[0;32m   1108\u001b[0m     \u001b[38;5;66;03m# calculate fprime(x0) first, as this may be cached by ScalarFunction\u001b[39;00m\n\u001b[0;32m   1109\u001b[0m     f1 \u001b[38;5;241m=\u001b[39m fprime(\u001b[38;5;241m*\u001b[39m((x0,) \u001b[38;5;241m+\u001b[39m args))\n\u001b[1;32m-> 1110\u001b[0m     f2 \u001b[38;5;241m=\u001b[39m fprime(\u001b[38;5;241m*\u001b[39m((x0 \u001b[38;5;241m+\u001b[39m epsilon\u001b[38;5;241m*\u001b[39mp,) \u001b[38;5;241m+\u001b[39m args))\n\u001b[0;32m   1111\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m (f2 \u001b[38;5;241m-\u001b[39m f1) \u001b[38;5;241m/\u001b[39m epsilon\n",
      "\u001b[1;31mValueError\u001b[0m: operands could not be broadcast together with shapes (4,) (12,) "
     ]
    }
   ],
   "source": [
    "theta_min = res.x\n",
    "predictions = predict(theta_min, X)\n",
    "correct = [1 if ((a == 1 and b == 1) or (a == 0 and b == 0)) else 0 for (a, b) in zip(predictions, y)]\n",
    "accuracy = (sum(map(int, correct)) % len(correct))\n",
    "print ('accuracy = {0}%'.format(accuracy))\n",
    "print(classification_report(y, predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "luDLSnm3jlJL"
   },
   "source": [
    "我们的逻辑回归分类器预测正确，如果一个学生被录取或没有录取，达到89%的精确度。不坏！记住，这是训练集的准确性。我们没有保持住了设置或使用交叉验证得到的真实逼近，所以这个数字有可能高于其真实值（这个话题将在以后说明）。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "EO1hfP0j-0cc"
   },
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xIvApW8HjBZg"
   },
   "source": [
    "## 寻找决策边界\n",
    "决策便捷是一系列的$x$可以满足：$$\\frac{1}{1+e^{-\\theta^T x}}=0.5$$\n",
    "也就是说$$e^{-\\theta^T x} = 1$$\n",
    "意味着 $$\\theta^T X = 0$$\n",
    "\n",
    "这是一条直线。\n",
    "\n",
    "参考这里：http://stats.stackexchange.com/questions/93569/why-is-logistic-regression-a-linear-classifier\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oErM-l5djBZh"
   },
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'res' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[36], line 1\u001b[0m\n\u001b[1;32m----> 1\u001b[0m \u001b[38;5;28mprint\u001b[39m(res\u001b[38;5;241m.\u001b[39mx)\n",
      "\u001b[1;31mNameError\u001b[0m: name 'res' is not defined"
     ]
    }
   ],
   "source": [
    "print(res.x) # this is final theta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Bmtqg98tjBZk"
   },
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'res' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[37], line 5\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[38;5;66;03m# x0 = 1\u001b[39;00m\n\u001b[0;32m      2\u001b[0m \u001b[38;5;66;03m# 定义x1的range后，根据 θ0 + x1*θ1 + x2*θ2 = 0，可得x2\u001b[39;00m\n\u001b[0;32m      3\u001b[0m x1 \u001b[38;5;241m=\u001b[39m np\u001b[38;5;241m.\u001b[39marange(\u001b[38;5;241m130\u001b[39m, step\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m0.1\u001b[39m)\n\u001b[1;32m----> 5\u001b[0m coef \u001b[38;5;241m=\u001b[39m \u001b[38;5;241m-\u001b[39m(res\u001b[38;5;241m.\u001b[39mx \u001b[38;5;241m/\u001b[39m res\u001b[38;5;241m.\u001b[39mx[\u001b[38;5;241m2\u001b[39m])  \u001b[38;5;66;03m# find the equation\u001b[39;00m\n\u001b[0;32m      6\u001b[0m x2 \u001b[38;5;241m=\u001b[39m coef[\u001b[38;5;241m0\u001b[39m] \u001b[38;5;241m+\u001b[39m coef[\u001b[38;5;241m1\u001b[39m]\u001b[38;5;241m*\u001b[39mx\n",
      "\u001b[1;31mNameError\u001b[0m: name 'res' is not defined"
     ]
    }
   ],
   "source": [
    "# x0 = 1\n",
    "# 定义x1的range后，根据 θ0 + x1*θ1 + x2*θ2 = 0，可得x2\n",
    "x1 = np.arange(130, step=0.1)\n",
    "\n",
    "coef = -(res.x / res.x[2])  # find the equation\n",
    "x2 = coef[0] + coef[1]*x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zW-rRiS-jBZu"
   },
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "lmplot() got multiple values for argument 'data'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "Cell \u001b[1;32mIn[38], line 4\u001b[0m\n\u001b[0;32m      1\u001b[0m \u001b[38;5;28;01mimport\u001b[39;00m \u001b[38;5;21;01mseaborn\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m \u001b[38;5;21;01msns\u001b[39;00m\n\u001b[0;32m      2\u001b[0m sns\u001b[38;5;241m.\u001b[39mset(context\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mnotebook\u001b[39m\u001b[38;5;124m\"\u001b[39m, style\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mticks\u001b[39m\u001b[38;5;124m\"\u001b[39m, font_scale\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m1.5\u001b[39m)\n\u001b[1;32m----> 4\u001b[0m sns\u001b[38;5;241m.\u001b[39mlmplot(\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mexam1\u001b[39m\u001b[38;5;124m'\u001b[39m, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mexam2\u001b[39m\u001b[38;5;124m'\u001b[39m, hue\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124madmitted\u001b[39m\u001b[38;5;124m'\u001b[39m, data\u001b[38;5;241m=\u001b[39mdata, \n\u001b[0;32m      5\u001b[0m            size\u001b[38;5;241m=\u001b[39m\u001b[38;5;241m6\u001b[39m, \n\u001b[0;32m      6\u001b[0m            fit_reg\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mFalse\u001b[39;00m, \n\u001b[0;32m      7\u001b[0m            scatter_kws\u001b[38;5;241m=\u001b[39m{\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124ms\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;241m25\u001b[39m}\n\u001b[0;32m      8\u001b[0m           )\n\u001b[0;32m     10\u001b[0m plt\u001b[38;5;241m.\u001b[39mplot(x1, x2, \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mgrey\u001b[39m\u001b[38;5;124m'\u001b[39m)\n\u001b[0;32m     11\u001b[0m plt\u001b[38;5;241m.\u001b[39mxlim(\u001b[38;5;241m0\u001b[39m, \u001b[38;5;241m130\u001b[39m)\n",
      "\u001b[1;31mTypeError\u001b[0m: lmplot() got multiple values for argument 'data'"
     ]
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "sns.set(context=\"notebook\", style=\"ticks\", font_scale=1.5)\n",
    "\n",
    "sns.lmplot('exam1', 'exam2', hue='admitted', data=data, \n",
    "           size=6, \n",
    "           fit_reg=False, \n",
    "           scatter_kws={\"s\": 25}\n",
    "          )\n",
    "\n",
    "plt.plot(x1, x2, 'grey')\n",
    "plt.xlim(0, 130)\n",
    "plt.ylim(0, 130)\n",
    "plt.title('Decision Boundary')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qgr6k1ybjlJM"
   },
   "source": [
    "## 正则化逻辑回归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "raPYPfoXjlJN"
   },
   "source": [
    "在训练的第二部分，我们将要通过加入正则项提升逻辑回归算法。如果你对正则化有点眼生，或者喜欢这一节的方程的背景，请参考在\"exercises\"文件夹中的\"ex2.pdf\"。简而言之，正则化是成本函数中的一个术语，它使算法更倾向于“更简单”的模型（在这种情况下，模型将更小的系数）。这个理论助于减少过拟合，提高模型的泛化能力。这样，我们开始吧。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8HRDKZsYjlJO"
   },
   "source": [
    "设想你是工厂的生产主管，你有一些芯片在两次测试中的测试结果。对于这两次测试，你想决定是否芯片要被接受或抛弃。为了帮助你做出艰难的决定，你拥有过去芯片的测试数据集，从其中你可以构建一个逻辑回归模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "JuiBlOvjjlJR"
   },
   "source": [
    "和第一部分很像，从数据可视化开始吧！"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "PPewmmIgjlJS"
   },
   "outputs": [],
   "source": [
    "data2 = pd.read_csv('ex2data2.txt', names=['test1', 'test2', 'accepted'])\n",
    "data2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "PH37JgMmjlJV"
   },
   "outputs": [],
   "source": [
    "positive = data2[data2['accepted'].isin([1])]\n",
    "negative = data2[data2['accepted'].isin([0])]\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(12,8))\n",
    "ax.scatter(positive['test1'], positive['test2'], s=50, c='b', marker='o', label='Accepted')\n",
    "ax.scatter(negative['test1'], negative['test2'], s=50, c='r', marker='x', label='Rejected')\n",
    "ax.legend()\n",
    "ax.set_xlabel('Test 1 Score')\n",
    "ax.set_ylabel('Test 2 Score')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "nKZFQDlUjlJY"
   },
   "source": [
    "哇，这个数据看起来可比前一次的复杂得多。特别地，你会注意到其中没有线性决策界限，来良好的分开两类数据。一个方法是用像逻辑回归这样的线性技术来构造从原始特征的多项式中得到的特征。让我们通过创建一组多项式特征入手吧。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "0WuhZaH9PyJv"
   },
   "outputs": [],
   "source": [
    "def feature_mapping(x, y, power, as_ndarray=False):\n",
    "#     \"\"\"return mapped features as ndarray or dataframe\"\"\"\n",
    "    # data = {}\n",
    "    # # inclusive\n",
    "    # for i in np.arange(power + 1):\n",
    "    #     for p in np.arange(i + 1):\n",
    "    #         data[\"f{}{}\".format(i - p, p)] = np.power(x, i - p) * np.power(y, p)\n",
    "\n",
    "    data = {\"f{}{}\".format(i - p, p): np.power(x, i - p) * np.power(y, p)\n",
    "                for i in np.arange(power + 1)\n",
    "                for p in np.arange(i + 1)\n",
    "            }\n",
    "\n",
    "    if as_ndarray:\n",
    "        return pd.DataFrame(data).as_matrix()\n",
    "    else:\n",
    "        return pd.DataFrame(data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sISiRom-jlJZ"
   },
   "outputs": [],
   "source": [
    "degree = 5\n",
    "x1 = np.array(df.test1)\n",
    "x2 = np.array(df.test2)\n",
    "\n",
    "data = feature_mapping(x1, x2, power=6)\n",
    "print(data.shape)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UaNs-uSEjlJe"
   },
   "source": [
    "现在，我们需要修改第1部分的成本和梯度函数，包括正则化项。首先是成本函数："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "YgE_bAV2jlJe"
   },
   "source": [
    "## Regularized cost（正则化代价函数）\n",
    "$$J\\left( \\theta  \\right)=\\frac{1}{m}\\sum\\limits_{i=1}^{m}{[-{{y}^{(i)}}\\log \\left( {{h}_{\\theta }}\\left( {{x}^{(i)}} \\right) \\right)-\\left( 1-{{y}^{(i)}} \\right)\\log \\left( 1-{{h}_{\\theta }}\\left( {{x}^{(i)}} \\right) \\right)]}+\\frac{\\lambda }{2m}\\sum\\limits_{j=1}^{n}{\\theta _{j}^{2}}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "qBnOVnA9RI1v"
   },
   "outputs": [],
   "source": [
    "theta = np.zeros(data.shape[1])\n",
    "X = feature_mapping(x1, x2, power=6, as_ndarray=True)\n",
    "print(X.shape)\n",
    "\n",
    "y = get_y(df)\n",
    "print(y.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Kkw1EX4DjlJg"
   },
   "outputs": [],
   "source": [
    "def regularized_cost(theta, X, y, l=1):\n",
    "#     '''you don't penalize theta_0'''\n",
    "    theta_j1_to_n = theta[1:]\n",
    "    regularized_term = (l / (2 * len(X))) * np.power(theta_j1_to_n, 2).sum()\n",
    "\n",
    "    return cost(theta, X, y) + regularized_term\n",
    "#正则化代价函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "207-7syrRNE_"
   },
   "outputs": [],
   "source": [
    "regularized_cost(theta, X, y, l=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "5nkQHdeGRtF2"
   },
   "source": [
    "## Regularized gradient(正则化梯度) \n",
    "\n",
    "$$\\frac{\\partial J\\left( \\theta  \\right)}{\\partial {{\\theta }_{j}}}=\\left( \\frac{1}{m}\\sum\\limits_{i=1}^{m}{\\left( {{h}_{\\theta }}\\left( {{x}^{\\left( i \\right)}} \\right)-{{y}^{\\left( i \\right)}} \\right)} \\right)+\\frac{\\lambda }{m}{{\\theta }_{j}}\\text{ }\\text{             for  j}\\ge \\text{1}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "1BnWF1tEjlJj"
   },
   "source": [
    "请注意等式中的\"reg\" 项。还注意到另外的一个“学习率”参数。这是一种超参数，用来控制正则化项。现在我们需要添加正则化梯度函数："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ycKcQ0ndjlJk"
   },
   "source": [
    "如果我们要使用梯度下降法令这个代价函数最小化，因为我们未对${{\\theta }_{0}}$ 进行正则化，所以梯度下降算法将分两种情形：\n",
    "\\begin{align}\n",
    "  & Repeat\\text{ }until\\text{ }convergence\\text{ }\\!\\!\\{\\!\\!\\text{ } \\\\ \n",
    " & \\text{     }{{\\theta }_{0}}:={{\\theta }_{0}}-a\\frac{1}{m}\\sum\\limits_{i=1}^{m}{[{{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}}]x_{_{0}}^{(i)}} \\\\ \n",
    " & \\text{     }{{\\theta }_{j}}:={{\\theta }_{j}}-a\\frac{1}{m}\\sum\\limits_{i=1}^{m}{[{{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}}]x_{j}^{(i)}}+\\frac{\\lambda }{m}{{\\theta }_{j}} \\\\ \n",
    " & \\text{          }\\!\\!\\}\\!\\!\\text{ } \\\\ \n",
    " & Repeat \\\\ \n",
    "\\end{align}\n",
    "\n",
    "对上面的算法中 j=1,2,...,n 时的更新式子进行调整可得： \n",
    "${{\\theta }_{j}}:={{\\theta }_{j}}(1-a\\frac{\\lambda }{m})-a\\frac{1}{m}\\sum\\limits_{i=1}^{m}{({{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}})x_{j}^{(i)}}$\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cwxZ64B4jlJm"
   },
   "outputs": [],
   "source": [
    "def regularized_gradient(theta, X, y, l=1):\n",
    "#     '''still, leave theta_0 alone'''\n",
    "    theta_j1_to_n = theta[1:]\n",
    "    regularized_theta = (l / len(X)) * theta_j1_to_n\n",
    "\n",
    "    # by doing this, no offset is on theta_0\n",
    "    regularized_term = np.concatenate([np.array([0]), regularized_theta])\n",
    "\n",
    "    return gradient(theta, X, y) + regularized_term"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "m3mVnObqjlJo"
   },
   "source": [
    "就像在第一部分中做的一样，初始化变量。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "SQJGX8C5jlJq"
   },
   "outputs": [],
   "source": [
    "regularized_gradient(theta, X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "3HDWjlJrjlJs"
   },
   "source": [
    "让我们初始学习率到一个合理值。，果有必要的话（即如果惩罚太强或不够强）,我们可以之后再折腾这个。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "Ra_Rd5sFjlJs"
   },
   "outputs": [],
   "source": [
    "learningRate = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "c3oj6HgkjlJv"
   },
   "source": [
    "现在，让我们尝试调用新的默认为0的theta的正则化函数，以确保计算工作正常。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "haAzSuP3jlJw"
   },
   "outputs": [],
   "source": [
    "regularized_cost(theta2, X2, y2, learningRate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Co_DCQqPjlJ8"
   },
   "source": [
    "现在我们可以使用和第一部分相同的优化函数来计算优化后的结果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "twdWMetWSeAd"
   },
   "source": [
    "## 拟合参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "rb6a5ASMSg09"
   },
   "outputs": [],
   "source": [
    "import scipy.optimize as opt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "AXbKitbDSiqS"
   },
   "outputs": [],
   "source": [
    "print('init cost = {}'.format(regularized_cost(theta, X, y)))\n",
    "\n",
    "res = opt.minimize(fun=regularized_cost, x0=theta, args=(X, y), method='Newton-CG', jac=regularized_gradient)\n",
    "res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UyNK7mUXjlJ_"
   },
   "source": [
    "最后，我们可以使用第1部分中的预测函数来查看我们的方案在训练数据上的准确度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "8D7uUrYGSrW6"
   },
   "source": [
    "## 预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "koAbf3UhStRU"
   },
   "outputs": [],
   "source": [
    "final_theta = res.x\n",
    "y_pred = predict(final_theta, X)\n",
    "\n",
    "print(classification_report(y, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_yLVQLTFjlKI"
   },
   "source": [
    "虽然我们实现了这些算法，值得注意的是，我们还可以使用高级Python库像scikit-learn来解决这个问题。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cih8ekDVjlKK"
   },
   "outputs": [],
   "source": [
    "from sklearn import linear_model#调用sklearn的线性回归包\n",
    "model = linear_model.LogisticRegression(penalty='l2', C=1.0)\n",
    "model.fit(X2, y2.ravel())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "jaT8naLtjlKM"
   },
   "outputs": [],
   "source": [
    "model.score(X2, y2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "A-SkdKuQjlKP"
   },
   "source": [
    "这个准确度和我们刚刚实现的差了好多，不过请记住这个结果可以使用默认参数下计算的结果。我们可能需要做一些参数的调整来获得和我们之前结果相同的精确度。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ioOB1lqPUlp7"
   },
   "source": [
    "## 使用不同的 $\\lambda$ \n",
    "### 画出决策边界\n",
    "* 我们找到所有满足 $X\\times \\theta = 0$ 的x\n",
    "* instead of solving polynomial equation, just create a coridate x,y grid that is dense enough, and find all those $X\\times \\theta$ that is close enough to 0, then plot them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "wgDzdelZjBa6"
   },
   "outputs": [],
   "source": [
    "def draw_boundary(power, l):\n",
    "#     \"\"\"\n",
    "#     power: polynomial power for mapped feature\n",
    "#     l: lambda constant\n",
    "#     \"\"\"\n",
    "    density = 1000\n",
    "    threshhold = 2 * 10**-3\n",
    "\n",
    "    final_theta = feature_mapped_logistic_regression(power, l)\n",
    "    x, y = find_decision_boundary(density, power, final_theta, threshhold)\n",
    "\n",
    "    df = pd.read_csv('ex2data2.txt', names=['test1', 'test2', 'accepted'])\n",
    "    sns.lmplot('test1', 'test2', hue='accepted', data=df, size=6, fit_reg=False, scatter_kws={\"s\": 100})\n",
    "\n",
    "    plt.scatter(x, y, c='R', s=10)\n",
    "    plt.title('Decision boundary')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "SK7PmPjkjBa-"
   },
   "outputs": [],
   "source": [
    "def feature_mapped_logistic_regression(power, l):\n",
    "#     \"\"\"for drawing purpose only.. not a well generealize logistic regression\n",
    "#     power: int\n",
    "#         raise x1, x2 to polynomial power\n",
    "#     l: int\n",
    "#         lambda constant for regularization term\n",
    "#     \"\"\"\n",
    "    df = pd.read_csv('ex2data2.txt', names=['test1', 'test2', 'accepted'])\n",
    "    x1 = np.array(df.test1)\n",
    "    x2 = np.array(df.test2)\n",
    "    y = get_y(df)\n",
    "\n",
    "    X = feature_mapping(x1, x2, power, as_ndarray=True)\n",
    "    theta = np.zeros(X.shape[1])\n",
    "\n",
    "    res = opt.minimize(fun=regularized_cost,\n",
    "                       x0=theta,\n",
    "                       args=(X, y, l),\n",
    "                       method='TNC',\n",
    "                       jac=regularized_gradient)\n",
    "    final_theta = res.x\n",
    "\n",
    "    return final_theta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GqbGlpA7jBbB"
   },
   "outputs": [],
   "source": [
    "def find_decision_boundary(density, power, theta, threshhold):\n",
    "    t1 = np.linspace(-1, 1.5, density)\n",
    "    t2 = np.linspace(-1, 1.5, density)\n",
    "\n",
    "    cordinates = [(x, y) for x in t1 for y in t2]\n",
    "    x_cord, y_cord = zip(*cordinates)\n",
    "    mapped_cord = feature_mapping(x_cord, y_cord, power)  # this is a dataframe\n",
    "\n",
    "    inner_product = mapped_cord.as_matrix() @ theta\n",
    "\n",
    "    decision = mapped_cord[np.abs(inner_product) < threshhold]\n",
    "\n",
    "    return decision.f10, decision.f01\n",
    "#寻找决策边界函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "JZWGQjWljBbJ",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "draw_boundary(power=6, l=1)#lambda=1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "CLBBoc6ZjBbN"
   },
   "outputs": [],
   "source": [
    "draw_boundary(power=6, l=0)  # no regularization, over fitting，#lambda=0,没有正则化，过拟合了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zkOZ5XMZjBbS"
   },
   "outputs": [],
   "source": [
    "draw_boundary(power=6, l=100)  # underfitting，#lambda=100,欠拟合"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "ML-Exercise2.ipynb",
   "provenance": [],
   "toc_visible": true,
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
